HTTP Error 500 on Mac Mini M2 8 GB Qwen 3.5 9B

[jaudo]

Trying to run this repo
Mac Mini M2 8 GB
Qwen 3.5 9B Q4
Context size: 4096

HTTP Error 500. Any idea?

[walter-grace]

Qwen 3.5 9B Q4

try a lower bit - double check if you have claude or something else taking up your RAM.

ask claude if anything is taking up the space on your RAM

UD-IQ2_XXS
3.19 GB

UD-IQ2_M
3.65 GB

UD-Q2_K_XL
4.12 GB

[jaudo]

I was able to run it, by disabling the GPU layers, got 10 t/s running by CPU only. On smaller Quants also works. Also the 4B model running at 20 t/s with GPU layers enabled.