NVIDIA-Nemotron-3.5-Bolt-E4B-oQ4e-fp16-0.7.3.dev2
This model was quantized using oQ (oMLX v0.7.3.dev2) mixed-precision quantization.
The effective average is 6.324 bits/weight for the complete checkpoint.
Quantization and Bit Distribution
| Storage format | Logical weights | Share of total | Tensor storage | Effective bits/weight |
|---|---|---|---|---|
| Affine 4-bit, group 64 | 6.150B | 76.917% | 3.222 GiB | 4.50 |
| Affine 5-bit, group 64 | 1.151B | 14.400% | 0.737 GiB | 5.50 |
| Affine 6-bit, group 64 | 0.222B | 2.770% | 0.168 GiB | 6.50 |
| FP16 | 0.001B | 0.007% | 0.001 GiB | 16.00 |
| FP32 | 0.472B | 5.905% | 1.759 GiB | 32.00 |
| Total | 7.996B | 100% | 5.887 GiB | 6.324 |
Storage Breakdown
| Component | Logical weights | Storage |
|---|---|---|
| Language model | 7.524B | 4.432 GB / 4.128 GiB |
| Vision | 0.167B | 0.669 GB / 0.623 GiB |
| Audio | 0.305B | 1.219 GB / 1.136 GiB |
| Total | 7.996B | 6.321 GB / 5.887 GiB |
- Downloads last month
- 98
Model size
8B params
Tensor type
F32
·
U32 ·
F16 ·
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for RepublicOfKorokke/NVIDIA-Nemotron-3.5-Bolt-E4B-oQ4e-fp16-0.7.3.dev2
Base model
google/gemma-4-E4B Finetuned
google/gemma-4-E4B-it Finetuned
nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16