Skip to content

Extract a shared routed-expert FFN helper and fix the fmt=6 input rotation #4

Description

@lineape

What to build

Eliminate the parallel FFN copy — the root of the maintainer's objection and the fmt=6 bug — by extracting the four-step routed-expert FFN (expert_gate_up → siluf → e8_rot_rows → matmul_qt) into ONE shared helper called by both moe()'s CPU path and cluster_worker_run, and applying the shared E8_XE input rotation on both sides. Do NOT patch the worker's copy in place; do NOT introduce a new parallel implementation ("two implementations of the same thing can't both land").

This fixes the missing fmt=6 input pre-rotation by construction: the worker feeds raw x to gate/up today, while moe() rotates the input first (E8_XE, guarded on g.fmt==6), then rotates the intermediate before the down matmul. Sharing one helper makes that drift impossible.

Acceptance criteria

Environment: codegraph-indexed — run codegraph explore first; refs in /home/eli/dev/_reference-docs-and-repos/colibri.

Metadata

Metadata

Assignees

No one assigned

    Labels

    effort/highNeeds frontier — new modules/core-kernel/complex state machinesready-for-agentFully specified, AFK-ready

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions