Request: Qwen3-30B-A3B-Instruct-2507-GPTQ-Int4
#1
by
jart25
- opened
Hi! Would it be possible to publish this variant, please? It would be greatly appreciated. Many thanks for your time QuantTrio!
The loss of Int4 quantization is quite significant.
Same request, please? Would be greatly appreciated. Many thank's..
@koesn posted a version here:
https://huggingface.co/jart25/Qwen3-30B-A3B-Instruct-2507-Autoround-Int-4bit-gptq
jart25
changed discussion status to
closed
It doesn't work. It said group size 32, where LMDeploy only support group size 128.
Our models are built with group size 128