Skip to content

[vllm-omni]: Omni Quant Support #1507

Description

@thuang6

Feature Description

This epic is created to track Omni model (vllm-omni targeted) quantization support status.
main dtypes includes: INT4, MXFP8, MXFP4 is future target

Motivation and Use Case

omni model enable status:

model architecture owner INT4 MXFP8
Qwen3-omni Qwen3OmniMoeForConditionalGeneration Liang ok
Qwen2.5-omni Qwen2_5OmniForConditionalGeneration Liang ok
GLM-Image GlmImageForConditionalGeneration GlmImageTransformer2DModel Liang ok
BAGEL-7B-MoT BagelForConditionalGeneration Liang ok
Ovis-Image OvisImageTransformer2DModel Liang ok
HunyuanImage-3.0 HunyuanImage3ForCausalMM Chang ok
OmniGen2 - Chang WIP
Z-Image ZImageTransformer2DModel Xin ok
Next-Step NextStep Xin WIP (Feature request in vllm-omni)
LongCat-Image LongCatImagePipeline Xin WIP
MiMo-Audio-7B-Instruct MiMoAudioForCausalLM Weiwei ok
Qwen3-TTS-12Hz-1.7B-CustomVoice Qwen3TTSForConditionalGeneration Weiwei ok
stable-audio-open - Weiwei ok
FLUX.1-dev FluxTransformer2DModel Mengni ok
SD - Mengni -
Wan WanTransformer3DModel Mengni/Liang ok
QwenImage - Mengni -

vllm-omni Support Status

vllm-project/vllm-omni#1325

HF model id Type status owner PR
black-forest-labs/FLUX.1-dev text→image Merged Yi #1777
Qwen/Qwen3-Omni-30B-A3B-Instruct text+audio→text+audio Merged Liang #2670
Qwen/Qwen2.5-Omni-7B text+audio→text+audio Merged Liang #2670
Qwen/Qwen-Image text→image Merged Yi #3588
Wan-AI/Wan2.2-T2V-A14B-Diffusers text→video Merged Liang #3353
Wan-AI/Wan2.2-I2V-A14B-Diffusers image→video Merged Liang #3353
Wan-AI/Wan2.2-S2V-14B audio+image+text→video WIP Liang
zai-org/GLM-Image text→image Merged Liang #3059

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions