Add K3 three-stage RL path: proxy checkpoint tooling, RL data prep, and fused-layout checkpoint cleanup - #20
Draft
ISEEKYAN wants to merge 18 commits into
Draft
Add K3 three-stage RL path: proxy checkpoint tooling, RL data prep, and fused-layout checkpoint cleanup#20ISEEKYAN wants to merge 18 commits into
ISEEKYAN wants to merge 18 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The implementation side of the K3 three-stage RL path, cut fresh from
mainwith no experiment scaffolding.lite/checkpoint.py: the fused attention input projection is no longer concatenated with self-computed offsets, and per-projection semantics are no longer derived by matching substrings of the parameter name. The layout is declared and the offsets are derived from that declaration.lite/fused_layout.py: K3 now owns the minimal row-axis checkpoint layout declarations it needs, removing the dependency on the unmerged MLite fused-weights module.lite/protocol.py: K3 owns decoder-layer selection for R3 router replay; this folds the former Fix K3 R3 router discovery with current MLite #21 into this PR.lite/model.py,config.py: the declaration surface (per-projection name, head count, head dim, order) and the config plumbing behind it.primitive/router.py: explicit FP32 gating contract for the shared router.tools/build_proxy_checkpoint.py,tools/prepare_gsm8k_rl_from_sft.py: reproducible construction of the proxy checkpoint and of the RL dataset from the SFT split.Why
Weight synchronisation into the inference engine was producing a deterministic, silent corruption of one segment of the fused seven-way projection: the same element index and the same bad-element count reproduced byte-for-byte across independent runs, while the source-side tensor was entirely finite. Hand-computed fusion offsets and name-substring parsing were the mechanism; declaring the layout removes the class of bug rather than one instance of it.
Keeping the K3-specific layout declaration in this package also removes the import closure that previously required an unmerged MLite PR.
Validation
megatron.lite.primitive.quantization.qatand differs in its MXFP4 export contract. These paths were not changed by this update.Note
Opened as a draft: the GPU re-verification of the synchronisation path has not run yet.