Skip to content

refactor(diffusion): centralize sampling and replay loops - #29

Draft
leviking98z-rgb wants to merge 2 commits into
refactor/model-plugin-sdkfrom
refactor/diffusion-runner
Draft

refactor(diffusion): centralize sampling and replay loops#29
leviking98z-rgb wants to merge 2 commits into
refactor/model-plugin-sdkfrom
refactor/diffusion-runner

Conversation

@leviking98z-rgb

@leviking98z-rgb leviking98z-rgb commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Summary

  • add model-neutral DiffusionRunner and VideoDiffusionRunner implementations for schedule validation, initial-latent handling, sparse trajectory capture, sampling, replay, autocast/precision policy, single-step prediction, and the default trainable surface
  • place diffusion-specific contracts and control flow under unirl/models/diffusion/, while retaining the pre-existing unirl.models.types.diffusion and unirl.models.types.replay_result paths as compatibility facades
  • migrate the ten ordinary image/video stages: SD3, Z-Image, Qwen-Image (including Edit-Plus by inheritance), Boogu-Image, Flux.2-Klein, WAN 2.1/2.2, HunyuanVideo 1.0/1.5, and HunyuanImage 3
  • preserve model-specific behavior through narrow hooks for SD3 batched replay, Qwen packed geometry, Boogu CFG range/RoPE, Flux eval mode, WAN 2.2 secondary guidance, HunyuanVideo 1.5 vision constants, and HI3 late-bound noise/state
  • keep BAGEL packed/Navit replay and LTX2 joint audio-video SDE as explicit transition-kernel exceptions, enforced by the stdlib framework guard
  • update the contributor contract to route future ordinary diffusion models through the shared model-domain package

The layer is a net reduction of 1,667 lines against refactor/model-plugin-sdk while retaining concrete stage dotpaths and public stage methods.

Related Issue

Related to Tencent-Hunyuan#285.

Test Plan

  • SKIP=no-commit-to-branch pre-commit run --all-files --show-diff-on-failure
  • python lint/check_framework_contracts.py
  • python lint/check_architecture_boundaries.py
  • python lint/check_experimental_boundaries.py
  • python lint/check_recipe_targets.py
  • python -m compileall -q unirl lint experimental/refl/models/types.py
  • one-off Python 3.12 import/identity check: canonical unirl.models.diffusion exports and the two retained unirl.models.types compatibility facades resolve to the same classes
  • one-off Python 3.12 CPU fake-kernel harness: generic image/video sampling, sparse trajectory storage, replay, prediction, trainable surface, precision, and modality passed after the package move
  • earlier one-off Python 3.12 CPU fake-kernel harness: sampling, sparse trajectory storage, replay/subset replay, prediction, trainable surface, precision, and modality passed for all 10 migrated stages
  • earlier one-off model-hook checks: WAN 2.2 guidance fallback, HunyuanVideo 1.5 vision kwargs, HI3 state isolation and deterministic late-bound NoiseRecipe, Boogu CFG-range/RoPE cache, and Flux replay mode restoration passed
  • earlier one-off SD3 FlowSDEStrategy harness: serial and one-forward batched replay produced identical log-probs and previous-sample means
  • earlier one-off Hydra composition harness: 120/120 examples/**/*.yaml recipes composed and resolved
  • GPU checkpoint/training parity not run; reason: this environment has no accelerator or model checkpoints. Numerical transformer/SDE kernels are unchanged, but a real-checkpoint smoke remains appropriate before merging.

Compatibility / Risk

  • No recipe schema, _target_ path, checkpoint, dataset, request, or LatentSegment format changes.
  • Concrete stages inherit the same diffuse, replay, predict_noise_at_step, and trainable_module surface from the runner. HI3 additionally accepts the common optional initial_latents argument.
  • Existing imports from unirl.models.types.diffusion and unirl.models.types.replay_result remain valid. The temporary unirl.models.types.diffusion_runner path existed only inside this unmerged Draft stack and is replaced by the canonical unirl.models.diffusion package.
  • Replay now consistently places stored trajectory tensors on bundle.device; this preserves existing SD3/Z-Image/Boogu dedicated-engine behavior and makes the same CPU-returned trajectory path available to the other ordinary stages.
  • Video stages use one centralized 6D replay check and modality=VIDEO factory. BAGEL and LTX2 retain their specialized runners.
  • This is a stacked Draft PR based on refactor/model-plugin-sdk; review commits 317d698 and 747f112 for this layer.

Reviewer Notes

Checklist

  • I reviewed the changed code and removed unrelated/generated artifacts.
  • I updated tests, docs, and configs where needed, or explained why not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant