gemma-4-E4B-it-qat-q4_0-unquantized-oQ4e-fp16-0.7.3.dev2
This model was quantized using oQ (oMLX v0.7.3.dev2) mixed-precision quantization.
The effective average is 6.335 bits/weight for the complete checkpoint.
Quantization and Bit Distribution
| Storage format | Logical weights | Share of total | Tensor storage | Effective bits/weight |
|---|---|---|---|---|
| Affine 4-bit, group 64 | 6.107B | 76.906% | 3.199 GiB | 4.50 |
| Affine 5-bit, group 64 | 1.139B | 14.343% | 0.729 GiB | 5.50 |
| Affine 6-bit, group 64 | 0.222B | 2.798% | 0.168 GiB | 6.50 |
| FP16 | 0.001B | 0.007% | 0.001 GiB | 16.00 |
| FP32 | 0.472B | 5.946% | 1.759 GiB | 32.00 |
| Total | 7.941B | 100% | 5.857 GiB | 6.335 |
Storage Breakdown
| Component | Logical weights | Storage |
|---|---|---|
| Language model | 7.469B | 4.400 GB / 4.098 GiB |
| Vision | 0.167B | 0.669 GB / 0.623 GiB |
| Audio | 0.305B | 1.219 GB / 1.136 GiB |
| Total | 7.941B | 6.289 GB / 5.857 GiB |
- Downloads last month
- 279
Model size
8B params
Tensor type
F32
·
U32 ·
F16 ·
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for RepublicOfKorokke/gemma-4-E4B-it-qat-q4_0-unquantized-oQ4e-fp16-0.7.3.dev2
Base model
google/gemma-4-E4B Finetuned
google/gemma-4-E4B-it