Intel/Ling-flash-2.0-gguf-q2ks-mixed-AutoRound
103B
•
Updated
•
287
•
5
wenhua cheng
wenhuach
AI & ML interests
Model Compression, CV
Recent Activity
updated
a model
about 4 hours ago
Intel/Ling-flash-2.0-gguf-q2ks-mixed-AutoRound
posted
an
update
3 days ago
🚀 AutoRound(https://github.com/intel/auto-round) is now supported by SGLang!
After integrations with TorchAO, Transformers, and VLLM, AutoRound-quantized models are now officially compatible with SGLang — bringing faster and more flexible deployment to your LLM workflows.
💡 We’ve also enhanced the RTN mode (--iters 0), cutting quantization costs significantly for low-resource users.
⭐ Star our repo and stay tuned for more exciting updates!
reacted
to
their
post
with 🚀
3 days ago
AutoRound keeps evolving its LLM quantization algorithm! 🚀
After enhancing W2A16 quantization, we now offer a fast algorithm to generate mixed bits/data-type schemes (~2mins for 8B models), great for MXFP4 and W2A16.
Learn more: https://github.com/intel/auto-round/blob/main/docs/step_by_step.md#autoscheme
Organizations
updated
a
model
about 4 hours ago
posted
an
update
3 days ago
Post
159
🚀 AutoRound(https://github.com/intel/auto-round) is now supported by SGLang!
After integrations with TorchAO, Transformers, and VLLM, AutoRound-quantized models are now officially compatible with SGLang — bringing faster and more flexible deployment to your LLM workflows.
💡 We’ve also enhanced the RTN mode (--iters 0), cutting quantization costs significantly for low-resource users.
⭐ Star our repo and stay tuned for more exciting updates!
After integrations with TorchAO, Transformers, and VLLM, AutoRound-quantized models are now officially compatible with SGLang — bringing faster and more flexible deployment to your LLM workflows.
💡 We’ve also enhanced the RTN mode (--iters 0), cutting quantization costs significantly for low-resource users.
⭐ Star our repo and stay tuned for more exciting updates!
reacted to
their
post with 🚀
3 days ago
Post
1696
AutoRound keeps evolving its LLM quantization algorithm! 🚀
After enhancing W2A16 quantization, we now offer a fast algorithm to generate mixed bits/data-type schemes (~2mins for 8B models), great for MXFP4 and W2A16.
Learn more: https://github.com/intel/auto-round/blob/main/docs/step_by_step.md#autoscheme
After enhancing W2A16 quantization, we now offer a fast algorithm to generate mixed bits/data-type schemes (~2mins for 8B models), great for MXFP4 and W2A16.
Learn more: https://github.com/intel/auto-round/blob/main/docs/step_by_step.md#autoscheme
Inference with llama.cpp + Open WebUI gives repeating `?`
4
#1 opened 6 days ago
by
whoisjeremylam
updated
a
model
10 days ago
published
a
model
10 days ago
posted
an
update
14 days ago
Post
1696
AutoRound keeps evolving its LLM quantization algorithm! 🚀
After enhancing W2A16 quantization, we now offer a fast algorithm to generate mixed bits/data-type schemes (~2mins for 8B models), great for MXFP4 and W2A16.
Learn more: https://github.com/intel/auto-round/blob/main/docs/step_by_step.md#autoscheme
After enhancing W2A16 quantization, we now offer a fast algorithm to generate mixed bits/data-type schemes (~2mins for 8B models), great for MXFP4 and W2A16.
Learn more: https://github.com/intel/auto-round/blob/main/docs/step_by_step.md#autoscheme
2507 Thinking model release
7
#4 opened 23 days ago
by
anjeysapkovski
updated
3
models
about 1 month ago
published
a
model
about 1 month ago
updated
4
models
about 1 month ago
Intel/Qwen3-Next-80B-A3B-Instruct-int4-AutoRound
Text Generation
•
Updated
•
1.92k
•
9
Intel/Qwen3-Next-80B-A3B-Instruct-int4-mixed-AutoRound
Text Generation
•
Updated
•
918
•
20
Intel/Qwen3-Next-80B-A3B-Thinking-int4-AutoRound
Text Generation
•
Updated
•
1.67k
•
6
Intel/Qwen3-Next-80B-A3B-Thinking-int4-mixed-AutoRound
Text Generation
•
Updated
•
926
•
29
Adding `transformers` as the library tag
#3 opened about 1 month ago
by
ariG23498
CPU only?
4
#2 opened about 1 month ago
by
jujutechnology
updated
a
model
about 2 months ago
published
a
model
about 2 months ago