Misc. bug: Gemma4 Reasoning stopped working after started new Gemma4-12B support

[MauroChimenti]

Name and Version

On the latest versions on all Gemma4 models version , the reasoning mode stopped working. Even with -rea on , always the reasoning stage missing. For the new Gemma4-12B i see a wrong max context window size too. 131000 in place of 256000

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

llama-server

Command line

Problem description & steps to reproduce

Just use -rea on option with llama-server with Gemma 4

First Bad Commit

No response

Relevant log output

Logs

[avkwok]

I also found this bug. I can't enable gemma-4-12b thinking even i use lmstudio working tag 9496 (lmstudio can enable gemma-4-12b thinking). lmstudio release note said they had fix gemma4 loading bug but don't know details.

[ggerganov]

@MauroChimenti Make sure you have enabled thinking in the WebUI #23434

[avkwok]

I tested (tag 9518) with enabled thinking in the WebUI. It is still not thinking.
The command line also shows "thinking = 1"

[gcp]

I think I see this too, should be easy to confirm with bisecting.

[mjentsch]

Looks like this affects more models, I can confirm the lack of the reasoning part for unsloth/Qwen3.6-35B-A3B-MTP-GGUF:Q8_0 but not for all chat clients. It seems specific for the embedded UI and was introduced from b9444 -> b9445.

To find the point of break I used

#!/bin/bash

git checkout tags/$1

cmake -B build --fresh -DGGML_CUDA=ON
cmake --build build --config Release -j

build/bin/llama-server --offline --port 8080 --parallel 1 --cache-reuse 64 --cache-ram 65536 --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 3 --split-mode tensor --ctx-size 131072 -hf unsloth/Qwen3.6-27B-MTP-GGUF:Q8_0 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.00

and used Empty cache and hard reset before I let it regenerate a simple question... get a Reasoning block with b9444 but not with b9445.

[gcp]

i see a wrong max context window size too. 131000 in place of 256000

That's probably a GGUF bug? Which one are you using?

[gcp]

Okay, I understand the problem: the "Developer" Menu now has an "Enable Thinking" toggle. However, just enabling that toggle doesn't enable thinking. You need to additionally select a thinking level with the lightbulb in the chat box.

With the bulb being there, the Developer menu option is probably misnamed/confusing? What does it actually control? Even when I uncheck it, the model will think when you use the lightbulb.

Edit: So the above is basically #24141

[ggerganov]

cc @allozaur

[fflurk]

Yes, #24141 is the exact same thing regarding the thinking, and exactly how I discovered it. In reality it had nothing to do with Gemma4-12B at all, just a unlucky release coincidence that an updated was needed for the new model as the same time of the merge of #23434
When troubleshooting with Antigravity, Gemini literally had to tell me to look closer at the screenshot I provided and I said "there is no such button" 😂

[fflurk]

@MauroChimenti Regarding the context size, Google had the wrong metadata in their model. Anyone that downloaded the model or any quantization before they fixed it will have the wrong context. https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/discussions/8
You don't have to re-download to fix though, you can easily patch the GGUF metadata yourself using the official gguf Python tools.

1. Install the gguf package:

pip install gguf

2. Patch the context length on your model:

gguf-set-metadata "path/to/your/model.gguf" gemma4.context_length 262144 --force

Once you run that, just restart llama-server and it will correctly boot up with the full 256k context window!.

Maybe there even is a way to do this within llama.cpp but that is at least one way.

[fflurk]

Regarding the context, I started a discussion here with information and a feature idea to flax/auto-patch the issue. Feel free to vote on it for visibility.
#24198

[coder543]

If possible, I would also prefer the reasoning mode to be "on" by default until the user changes it. The introduction of the reasoning toggle made it so reasoning is something you have to manually turn on, when I can't think of a single reasoning model that has "off" as the default in the template.

[githubsean]

I'm getting this on the Qwen 3.6 and Gemma 4 unsloth models. I haven't checked any developer or UI options. I tried forcing thinking on in the CLI.

I think if the model defines thinking by default, llama-server should follow that default.

[CMay]

@allozaur honestly I overlooked this button as well in the UI and did bisects to try to figure out when thinking broke. it does not look like a thinking button when it's disabled with the crossing line over it. it looked like a location marker, like for enabling or disabling GPS access to the model if you're on a laptop that supports that.

it's also nudged up against the model details which don't do anything, compared to the + in the lower left corner which does do something. it might make more sense to enlarge the icon and button to be about the same size as the + button in the lower left and put it next to the + button. it also already becomes orange highlighted when its active, so does the cross line need to be there obscuring the icon?