Post
3430
Released an AWQ quantized version of BosonAI’s Higgs-Llama-3-70B model! 🎉
The Higgs-Llama-3-70B is an LLM specialized in role-playing, useful for game characters.
Using an NVIDIA B200 GPU, I was able to compress the huge 140GB model into 37GB while keeping minimal perplexity 👍
ronantakizawa/higgs-llama-3-70b-awq
The Higgs-Llama-3-70B is an LLM specialized in role-playing, useful for game characters.
Using an NVIDIA B200 GPU, I was able to compress the huge 140GB model into 37GB while keeping minimal perplexity 👍
ronantakizawa/higgs-llama-3-70b-awq