Can you provide quantization like Qwen3-235B-A22B-IGPTQ-INT4 ?

#33
by djdeniro - opened

Can you provide quantization like Qwen3-235B-A22B-IGPTQ-INT4 ? it's only one way to launch in on amd gpu 8x with -tp 2 and -pp 4.

Thank you for any advice / help!

Sign up or log in to comment