PowerInfer

/

SmallThinker-21BA3B-Instruct-GGUF

like 34

Follow

PowerInfer 383

I only have 12GB VRAM (Nvidia Geforce RTX 3060) available right now. If you upload a Q3, i will try this model tomorrow with your pull-request in the server version of llama.cpp. (But don't expect much coding feedback from me. C++ and Python ain't my forte at present time.)