Image-Text-to-Text
Safetensors
Trellis
vllm
qwen3_5
exl3
exllamav3
mixed-precision
quantized
qwen3.8
vision-language
gilded-gnosis
conversational
4-bit precision
Instructions to use malaiwah/Qwen3.8-27B-EXL3-K5K6-hydrated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Trellis
How to use malaiwah/Qwen3.8-27B-EXL3-K5K6-hydrated with Trellis:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Fix credits: RTX 5090 tester is malaiwah (self), GLM-5.2-EXL3-TR3-MTP78 is malaiwah's checkpoint; jcartu contributed the recipe/runtime that informed it
Browse files
README.md
CHANGED
|
@@ -1806,9 +1806,8 @@ content-addressing showing through.
|
|
| 1806 |
- **[Martin Vit / voipmonitor / festr2](https://huggingface.co/voipmonitor)** ([local-inference-lab](https://github.com/local-inference-lab)) β the Gilded Gnosis vLLM fork, B12X Trellis kernel integration, the `voipmonitor/vllm` Docker images, the [rtx6kpro](https://github.com/local-inference-lab/rtx6kpro) serving wiki, and the `exl3-b6` online-encoding overlay whose distribution-fidelity protocol this evaluation follows.
|
| 1807 |
- **[Luke Alonso / lukealonso](https://huggingface.co/lukealonso)** β B12X kernel co-author (PR #31, NF3/NVFP4) and the NVFP4 quantization methodology whose protection patterns informed the recipe design.
|
| 1808 |
- **[Brandon Music / brandonmusic](https://huggingface.co/brandonmusic)** β the [ShapleyMCG](https://arxiv.org/abs/2607.12266) expert-bit allocation method, the EXL3 checkpoint patterns (TR3 schema, online K6, shared-expert handling) that this recipe family is modelled on, and the exllamav3 SM120 retile fork.
|
| 1809 |
-
- **[Josh Cartu / jcartu](https://huggingface.co/jcartu)** β the
|
| 1810 |
- **[Qwen](https://huggingface.co/Qwen)** β the base model ([Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B), Apache-2.0) and its official FP8 derivative ([Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8)).
|
| 1811 |
- **[nvidia/Qwen3.6-27B-NVFP4](https://huggingface.co/nvidia/Qwen3.6-27B-NVFP4)** β the role-aware recipe this design is modelled on (4-bit MLP, 8-bit attention, protected embeddings/vision/MTP).
|
| 1812 |
- **[unsloth/Qwen3.8-27B-NVFP4](https://huggingface.co/unsloth/Qwen3.8-27B-NVFP4)** β independent confirmation of the same protection pattern.
|
| 1813 |
-
- **[malaiwah](https://huggingface.co/malaiwah)** / [progressive-tensors](https://github.com/malaiwah/progressive-tensors) β research, recipe, harness and receipts. Per-expert EXL3 provenance work
|
| 1814 |
-
- An independent RTX 5090 tester β the context-capacity tables, and catching a 262K overclaim from a memory simulation.
|
|
|
|
| 1806 |
- **[Martin Vit / voipmonitor / festr2](https://huggingface.co/voipmonitor)** ([local-inference-lab](https://github.com/local-inference-lab)) β the Gilded Gnosis vLLM fork, B12X Trellis kernel integration, the `voipmonitor/vllm` Docker images, the [rtx6kpro](https://github.com/local-inference-lab/rtx6kpro) serving wiki, and the `exl3-b6` online-encoding overlay whose distribution-fidelity protocol this evaluation follows.
|
| 1807 |
- **[Luke Alonso / lukealonso](https://huggingface.co/lukealonso)** β B12X kernel co-author (PR #31, NF3/NVFP4) and the NVFP4 quantization methodology whose protection patterns informed the recipe design.
|
| 1808 |
- **[Brandon Music / brandonmusic](https://huggingface.co/brandonmusic)** β the [ShapleyMCG](https://arxiv.org/abs/2607.12266) expert-bit allocation method, the EXL3 checkpoint patterns (TR3 schema, online K6, shared-expert handling) that this recipe family is modelled on, and the exllamav3 SM120 retile fork.
|
| 1809 |
+
- **[Josh Cartu / jcartu](https://huggingface.co/jcartu)** β the MTP78 recipe and rank-sliced MTP runtime that informed [`malaiwah/GLM-5.2-EXL3-TR3-MTP78`](https://huggingface.co/malaiwah/GLM-5.2-EXL3-TR3-MTP78): the graft/encode/serve procedure for the MTP draft layer, plus the vLLM `normalize_rank_sliced_weight_name()` loader patch.
|
| 1810 |
- **[Qwen](https://huggingface.co/Qwen)** β the base model ([Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B), Apache-2.0) and its official FP8 derivative ([Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8)).
|
| 1811 |
- **[nvidia/Qwen3.6-27B-NVFP4](https://huggingface.co/nvidia/Qwen3.6-27B-NVFP4)** β the role-aware recipe this design is modelled on (4-bit MLP, 8-bit attention, protected embeddings/vision/MTP).
|
| 1812 |
- **[unsloth/Qwen3.8-27B-NVFP4](https://huggingface.co/unsloth/Qwen3.8-27B-NVFP4)** β independent confirmation of the same protection pattern.
|
| 1813 |
+
- **[malaiwah](https://huggingface.co/malaiwah)** / [progressive-tensors](https://github.com/malaiwah/progressive-tensors) β research, recipe, harness and receipts. Per-expert EXL3 provenance work, the per-bit error ladder, the [`GLM-5.2-EXL3-TR3-MTP78`](https://huggingface.co/malaiwah/GLM-5.2-EXL3-TR3-MTP78) checkpoint, and the RTX 5090 context-capacity qualification that caught a 262K overclaim.
|
|
|