[Experimental] Add NVFP4 E5M3 support and related components for model-free quantization

[xin3he]

Description

• Introduced NVFP4 E5M3 quantization scheme with high-precision QDQ weights.
• Implemented FakeActQuantLinear for runtime activation QDQ.
• Updated model-free compressor to handle NVFP4 E5M3 format.
• Enhanced FakeFormat to support NVFP4 E5M3 serialization.
• Added tests for NVFP4 E5M3 quantization and model conversion.

Update:

• support fake, auto_round, llm_compressor format
• support auto_round:fake, auto_round:torch_nvfp4_e5m3, auto_round:cute_nvfp4_e5m3 backend

Real Model Test [NVFP4_E5M3, Llama-3.1-8B-Instruct, lambada_openai]
1. BF16:
- time: 1min5s
- acc: 0.7213

2. auto_round format + auto_round:cute_nvfp4_e5m3 (dequant weight, qdq act)
- time: 1min48s
- acc: 0.7196

3. auto_round format + auto_round:torch_nvfp4_e5m3 (dequant weight, qdq act)
- time: 4min23s
- acc: 0.7204

4. llm_compressor format + auto_round:cute_nvfp4_e5m3 (dequant weight, qdq act)
- time: 1min48s
- acc: 0.7196

5. fake format + auto_round:fake (bf16 weight, qdq act)
- time: 3min 26
- acc: 0.7204

Type of Change

Experiment

Related Issues

Fixes or relates to #

Checklist Before Submitting

• My code has been tested locally.
• Documentation has been updated as needed.
• New or updated tests are included where applicable.
• The CUDA CI has passed. You can trigger it by commenting /azp run Unit-Test-CUDA-AutoRound.

[xin3he]

数值 root cause 是 CuTe QDQ 在 E5M3 block-scale 的 scale/mantissa 边界上与 PyTorch 参考实现有少量 ULP 级差异。这些差异经 32 层传播后可改变少数 token 排序，进而造成 LAMBADA 指标差别。它不是权重加载、GEMM、张量布局或评测随机性导致。

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[AutoRoundBot]

/azp run Unit-Test-CUDA-AutoRound

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

Adds experimental NVFP4 E5M3 (FP4 v2 + E5M3 scales) support across AutoRound’s export/inference/model-free paths, including a “fake” activation-QDQ runtime linear and an optional CUDA CuTe dispatch layer, plus corresponding unit tests.

Changes:

• Introduces NVFP4_E5M3 scheme preset and inference backends (torch_nvfp4_e5m3, cute_nvfp4_e5m3, fake) with backend selection/dispatch updates.
• Extends exporters (AutoRound + llm-compressor) and model-free compressor to serialize/pack NVFP4 E5M3 (including a “fake” format path).
• Adds CPU/CUDA tests validating packing/dequant/QDQ behavior and format resolution.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test/unit/test_cuda/quantization/test_nvfp4_e5m3_cute.py	Adds CUDA CuTe kernel correctness tests (QDQ + weight DQ).
test/unit/test_cpu/quantization/test_nvfp4_quant_linear.py	Adds CPU-side tests for NVFP4 E5M3 QuantLinear behaviors (fallbacks, caching, env override).
test/unit/test_cpu/quantization/test_model_free.py	Adds model-free NVFP4 E5M3 tests and FP8 scale hydration coverage.
test/unit/test_cpu/export/test_fake_format.py	Adds tests for fake-format save/load and Transformers load integration.
test/unit/test_cpu/core/test_format_decoupling.py	Extends format resolution tests for NVFP4_E5M3 backend/packing selection.
auto_round/utils/common.py	Monkey-patches Transformers compressed-tensors quantizer to route NVFP4 E5M3 through AutoRound qmodules.
auto_round/schemes.py	Registers the NVFP4_E5M3 preset scheme.
auto_round/inference/convert_model.py	Adds backend creation paths for NVFP4 E5M3 and “fake”; maps compressed-tensors NVFP4 E5M3 config to AutoRound inference config.
auto_round/inference/backend.py	Registers new backends and adds CuTe availability checker and dynamic imports.
auto_round/export/formats/base.py	Adds BackendDataType.NVFP4_E5M3 mapping to fp4_v2.
auto_round/export/formats/backends/llm_compressor.py	Adds NVFP4_E5M3 scheme support and fp4_v2 packing routing.
auto_round/export/formats/backends/fake.py	Enhances fake-format saving: unwrap wrapper layers and serialize quantization config safely.
auto_round/export/formats/backends/autoround.py	Adds NVFP4_E5M3 support and routes packing/export via llm-compressor fp packing helpers.
auto_round/export/export_to_llmcompressor/export_to_fp.py	Adds NVFP4_E5M3 scheme/format mapping and config initialization.
auto_round/export/export_to_llmcompressor/config.py	Adds initialize_nvfp4_e5m3_quantization() config builder.
auto_round/export/export_to_autoround/qlinear_fp.py	Adds packing support for fp4_v2 by encoding scales to E5M3 bytes.
auto_round/experimental/qmodules/nvfp4_e5m3.py	New NVFP4 E5M3 quantized Linear (PyTorch reference + optional CuTe dispatch).
auto_round/experimental/qmodules/fake.py	New FakeActQuantLinear: FP weights + activation QDQ at runtime.
auto_round/experimental/qmodules/init.py	Exposes newly added qmodules.
auto_round/data_type/nvfp.py	Adjusts reference FP4 quant path to avoid division-by-zero.
auto_round/compressors/model_free.py	Adds NVFP4_E5M3 model-free paths, compressed-tensors config building, FP8 scale hydration, and DeepSeek preprocessing extensions.
auto_round_extension/cuda/cute_nvfp4_e5m3.py	New optional CuTe DSL kernels for QDQ and weight dequantization (plus dispatch stubs).