Sem Karaman

me too, if I could fit it on my GPU. but I highly doubt it's the quantization. 3.5 35B and all sorts of merges of it works perfectly fine at pretty much any quant

i see, in that case if you already tried sampling (repeat penalty 1.05-1.1) and quant seems fine then only other possible fix that worked sometimes for me is a direct system prompt to 'not overthink' and 'allow knock on errors after first solution'

also i think it's a given that you should be running 0.2 temp for 'precise' responses

i would try Q8 as last resort

I see, this is usually because you are using Q4 quants locally? Try to go up to Q5 or Q8, reduce GPU layers and context length to fit into you VRAM.

oh 3.6 35B is a literal never ending reasoning loop for me. like 3 out 6 times need to kill the server type of deal

post your sampling setup