Skip to content

[LLMC][XPU][Llama-3.3-70B-Instruct][mxfp4]: save_pretrained OOM #1967

Description

@XuehaoSun

LOG

Traceback (most recent call last):
  File "/jenkins/17414/workspace/AutoRound_LLMC_example_test@2/llm-compressor/examples/autoround/quantization_w4a4_mxfp4/autoround_example.py", line 81, in <module>
    model.save_pretrained(SAVE_DIR, save_compressed=True)
  File "/opt/venv/lib/python3.12/site-packages/llmcompressor/transformers/compression/compressed_tensors_utils.py", line 74, in save_pretrained_wrapper
    compressor.compress_model(model)
  File "/opt/venv/lib/python3.12/site-packages/compressed_tensors/compressors/model_compressors/model_compressor.py", line 155, in compress_model
    compress_module(module, self.force_compression_format)
  File "/opt/venv/lib/python3.12/site-packages/compressed_tensors/compressors/base.py", line 193, in compress_module
    compressor.compress_module(module)
  File "/opt/venv/lib/python3.12/site-packages/compressed_tensors/compressors/base.py", line 109, in compress_module
    compressed_state_dict = cls.compress(state_dict, scheme)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/compressed_tensors/compressors/nvfp4/base.py", line 92, in compress
    state_dict["weight_packed"] = pack_fp4_to_uint8(quantized_weight)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/compressed_tensors/compressors/nvfp4/helpers.py", line 59, in pack_fp4_to_uint8
    abs_diff_x = torch.abs(abs_x.unsqueeze(-1) - kE2M1)  # [m, n, 8]
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: XPU out of memory. Tried to allocate 3.50 GiB. GPU 0 has a total capacity of 23.91 GiB of which 1.73 GiB is free. Of the allocated memory 17.79 GiB is allocated by PyTorch, and 647.73 MiB is reserved by PyTorch but unallocated. Please use `empty_cache` to release all unoccupied cached memory.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions