From 7cf4ae4d7b9293d888ee96469bdce715c7d8c9eb Mon Sep 17 00:00:00 2001 From: haonan3 Date: Thu, 30 Jul 2026 22:51:41 +0800 Subject: [PATCH] =?UTF-8?q?docs(algorithms):=20placement=20policy=20?= =?UTF-8?q?=E2=80=94=20core=20canon=20vs=20experimental=20variants=20(RFC)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Writes down the placement rules for loss algorithms agreed during the experimental-tier design: core keeps the canonical family bases and the team-flagship published algorithms; model-specific one-offs and unpublished variants live in the experimental package that needs them (selected via _target_ — trainers are unaffected either way); movement in either direction is a deliberate PR with config dotpaths migrated in the same change. No algorithms move in this PR — it is the policy RFC. --- unirl/algorithms/README.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/unirl/algorithms/README.md b/unirl/algorithms/README.md index 7201510ed..23fb4d7df 100644 --- a/unirl/algorithms/README.md +++ b/unirl/algorithms/README.md @@ -84,3 +84,26 @@ segment, expand advantages per token), keeping `supports_multi_update = False`. distribution; when unset it silently falls back to the `ARSamplingParams` default, *not* the engine's actual temperature, biasing every ratio with no raise. Watch `rollout_replay_logp_absdiff_mean` — it should be ~0 on an on-policy step. + +## Placement policy (RFC) + +`unirl/algorithms/` is a curated set, not an accumulation point. Where a +loss algorithm lives follows three rules: + +1. **Core keeps the canon**: the reference family bases (GRPO, FlowGRPO, + SFT losses) and the team-flagship published algorithms (CPPO, DRPO, + FlowDPPO — the repo's advertised surface). These carry core review and + stability guarantees. +2. **Variants start in `experimental/`**: model-specific one-offs and + unpublished experimental variants belong to the package that needs + them (`experimental//`), selected via `_target_` exactly like + core algorithms — the trainer never changes either way. New variants + default here first. +3. **Movement is a deliberate PR in either direction**: a variant + graduates into core when it becomes a recommended default (with its + config dotpaths migrated in the same PR — `check_recipe_targets` + gates stale paths); a core resident that stays model-specific or + unadopted is a candidate to move down. Never both copies at once. + +In-flight algorithm PRs choose their home under this policy at land +time. (RFC: comment on the placement PR if you want different lines.)