Skip to content

gguf-py: preserve MoE size labels for mmproj metadata#23618

Open
ooovenenoso wants to merge 2 commits into
ggml-org:masterfrom
ooovenenoso:fix/gguf-mmproj-moe-size-label
Open

gguf-py: preserve MoE size labels for mmproj metadata#23618
ooovenenoso wants to merge 2 commits into
ggml-org:masterfrom
ooovenenoso:fix/gguf-mmproj-moe-size-label

Conversation

@ooovenenoso
Copy link
Copy Markdown

@ooovenenoso ooovenenoso commented May 24, 2026

Fixes #23448.

For mmproj exports, total_params refers to the projector, while the parsed Hugging Face model id still refers to the parent model. This preserves the parent model size label for mmproj metadata so Gemma 4 projector exports do not turn the parent size into finetune text.

Covered cases:

  • google/gemma-4-26B-A4B-it with projector-sized total_params keeps general.size_label = 26B-A4B and general.finetune = it
  • google/gemma-4-31B-it with projector-sized total_params keeps general.size_label = 31B and general.finetune = it

Tested with:

PYTHONPATH=gguf-py python -m pytest gguf-py/tests/test_metadata.py -q
ruff check gguf-py/gguf/metadata.py gguf-py/tests/test_metadata.py conversion/base.py

AI assistance was used for code exploration and validation. I reviewed the change and understand the metadata parsing logic.

@ooovenenoso ooovenenoso requested a review from CISC as a code owner May 24, 2026 15:30
@github-actions github-actions Bot added the python python script changes label May 24, 2026
@dfriehs
Copy link
Copy Markdown
Contributor

dfriehs commented Jun 3, 2026

I don't think this is the right choice here, the mmproj for gemma-4-26B-A4B-it is not even close to 4B parameters. The ~500M parameters were simply close enough in this case to allow A4B to be used as the size label.

I think the mmproj should either always or never take the size label of the model it belongs to, no matter if it's number of parameters would allow part of the name as a label.

@ooovenenoso
Copy link
Copy Markdown
Author

Updated the PR based on the mmproj feedback:

  • The code now treats mmproj parsing separately: total_params is the projector size, so the parent model size label is preserved from the model id.
  • The test now uses a projector-sized value (576M) instead of 4B.
  • Added coverage for both reported Gemma 4 cases:
    • google/gemma-4-26B-A4B-it -> size_label = 26B-A4B, finetune = it
    • google/gemma-4-31B-it -> size_label = 31B, finetune = it

Verified locally:

PYTHONPATH=gguf-py python -m pytest gguf-py/tests/test_metadata.py -q
ruff check gguf-py/gguf/metadata.py gguf-py/tests/test_metadata.py conversion/base.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python python script changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misc. bug: gguf-py: size_label and finetune parsing for convert_hf_to_gguf.py broken for mmproj

2 participants