这次会不会有14B的AutoAWQ或者GPTQ？

by lingyezhixing - opened 23 days ago

23 days ago

快速翻译还是得靠vllm或者lmdeploy，llama.cpp的并发吞吐和prefill还是有点差

SakuraLLM org 22 days ago

正式版会有，测试版因为还在迭代中不能保证次次都有

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment