[Feature]: support svdquant for diffusion model

[chensuyue]

1. add SVDQuant algorithm for diffusion models. base version and residual iteration version.
2. export to nunchaku onefile format to use the mxfp4 svdquant kernel. (depend on [Task] Support SVDQuant MXFP4 kernel in Nunchaku. #2017)
3. MXFP4 recipes for Flux.1-dev.

[changwangss]

Feature: Add SVDQuant MXFP4 Support

Goal

Add a general SVDQuant workflow to AutoRound that supports MXFP4 quantization, QDQ evaluation, and Nunchaku-compatible export without depending on DeepCompressor or Nunchaku.

Modules

1. SVDQuant

   • Low-rank decomposition
   • Residual iterations and early stopping
   • Optional smooth calibration

2. MXFP4 Quantization

   • Reuse AutoRound RTN and SignRound
   • Support weight and activation QDQ
   • Align scale computation with Nunchaku

3. Export

   • Export a Nunchaku-compatible transformer
   • Support complete Diffusers pipelines with BF16 auxiliary components

4. Validation

   • Unit tests and CLI smoke tests
   • Quality comparison with BF16 and NVFP4

#2032

[changwangss]

bf16 flux.1-dev accuracy

+-------------+----------+
| CLIP        | 26.0189  |
+-------------+----------+
| CLIP-IQA    |  0.95436 |
+-------------+----------+
| ImageReward |  1.01834 |
+-------------+----------+

mxfp4 flux.1-dev command (nunchaku 4de4986 , auto-round f7df5f34)

CUDA_VISIBLE_DEVICES=6 nohup auto-round --model ./FLUX.1-dev --model_dtype bf16 --scheme MXFP4 --iters 200 --lr 0.005 --minmax_lr 0.005 --nsamples 128 --batch_size 1 --dataset ./coco2017-captions.tsv --num_inference_steps 50 --enable_svdquant --svdquant_residual_iters 20 --enable_svdquant_residual_early_stop --enable_svdquant_smooth --svdquant_smooth_num_grids 20 --svdquant_smooth_max_calibration_calls 256 --svdquant_low_rank_dtype bf16 --svdquant_model_adapter flux --format svdquant_nunchaku --disable_low_cpu_mem_usage --low_gpu_mem_usage --output_dir smooth_svdquant_signround > smooth_svdquant_signround.log 2>&1 &

mxfp4 flux.1-dev accuracy with smooth

+-------------+-----------+
| CLIP        | 26.1039   |
+-------------+-----------+
| CLIP-IQA    |  0.962655 |
+-------------+-----------+
| ImageReward |  1.02102  |
+-------------+-----------+

mxfp4 flux.1-dev command (nunchaku 4de4986 , auto-round f7df5f34)

CUDA_VISIBLE_DEVICES=6 nohup auto-round --model ./FLUX.1-dev --model_dtype bf16 --scheme MXFP4 --iters 200 --lr 0.005 --minmax_lr 0.005 --nsamples 128 --batch_size 1 --dataset ./coco2017-captions.tsv --num_inference_steps 50 --enable_svdquant --svdquant_residual_iters 20 --enable_svdquant_residual_early_stop  --svdquant_low_rank_dtype bf16 --svdquant_model_adapter flux --format svdquant_nunchaku --disable_low_cpu_mem_usage --low_gpu_mem_usage --output_dir smooth_svdquant_signround > smooth_svdquant_signround.log 2>&1 &

mxfp4 flux.1-dev accuracy with nosmooth

+-------------+-----------+
| CLIP        | 26.0727   |
+-------------+-----------+
| CLIP-IQA    |  0.959363 |
+-------------+-----------+
| ImageReward |  1.00238  |
+-------------+-----------+

nosmooth + svdquant + rtn

+-------------+-----------+
| CLIP        | 25.9624   |
+-------------+-----------+
| CLIP-IQA    |  0.946939 |
+-------------+-----------+
| ImageReward |  0.934579 |
+-------------+-----------+

smooth + svdquant + rtn

+-------------+-----------+
| CLIP        | 25.9719   |
+-------------+-----------+
| CLIP-IQA    |  0.947763 |
+-------------+-----------+
| ImageReward |  0.939392 |
+-------------+-----------+