[Bug] Hidream-dev produces noisy images on ROCm and Vulkan

[wbruna]

Git commit

0665a7f
(master-602-0665a7f)

Operating System & Version

Debian 13

GGML backends

HIP,Vulkan

Command-line arguments used

sd-cli -m HiDream-O1-Image-Dev-Q8_0.gguf -W 768 -H 768 -p "3D render of two black dice on a smooth surface" --cfg-scale 1 --steps 28

Steps to reproduce

The generation goes well until around ~2/3 of the steps, then noisy details begin to be introduced on the image. For instance, a preview around half of the inference:

And the final image:

In an attempt to use the original sigma scheduling of the distilled model, I've noticed it helps if low sigma values are avoided (not 100% sure about the exact values, but it doesn't seem to matter much):

sd-cli -m HiDream-O1-Image-Dev-Q8_0.gguf -W 768 -H 768 -p "3D render of two black dice on a smooth surface" --cfg-scale 1 --steps 19 --sigmas 0.999,0.987,0.974,0.96,0.945,0.929,0.913,0.895,0.877,0.857,0.836,0.814,0.79,0.764,0.737,0.707,0.675,0.64,0.602,0.0

The white reflections on the table were added only at the final step.

Skipped steps when using --cache-mode seem to help, and reduce the saturated look. Higher CFG values seem to help as well.

This happens consistently for any subject, but it's easier to notice on images that are supposed to be smooth (the model seemingly tries to compensate for the noise, adding more details to the image).

What you expected to happen

A smooth final image, as in the preview.

What actually happened

Noisy, saturated images.

Logs / error messages / stack trace

No response

Additional context / environment details

No response

Edit: after adjusting the quantization (#1496), I see similar outputs with Vulkan, too.

[wbruna]

Forgot to add: I've quantized the BF16 model myself with:
^model\.language_model\.layers\.[0-9]+\.(mlp\.down|self_attn\.o)_proj\.weight=q8_0,^model\.language_model\.layers\.[0-9]+\.(mlp\.(gate|up)|self_attn\.[kqv])_proj\.weight=q8_0

The same effect also happens on a straight q8_0 conversion.