Update README.md
Browse files
README.md
CHANGED
|
@@ -69,6 +69,14 @@ Q3_K_M | 11.5e9 | 5.58 | not tested, should work well
|
|
| 69 |
Q4_K_H | 12.5e9 | 5.49 | slightly smaller than IQ4_XS, similar performance
|
| 70 |
IQ4_XS | 12.9e9 | 5.38 | not tested, should work well
|
| 71 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
## Download the file from below:
|
| 73 |
| Link | Type | Size/e9 B | Notes |
|
| 74 |
|------|------|-----------|-------|
|
|
|
|
| 69 |
Q4_K_H | 12.5e9 | 5.49 | slightly smaller than IQ4_XS, similar performance
|
| 70 |
IQ4_XS | 12.9e9 | 5.38 | not tested, should work well
|
| 71 |
|
| 72 |
+
Usage:
|
| 73 |
+
|
| 74 |
+
This is a vision capable model. It can be used together with its multimedia projector layers to process images and text inputs
|
| 75 |
+
and generate text outputs. The mmproj file is made available in this repository. To test vision mode follow the docs in the mtmd
|
| 76 |
+
readme in the tools directory of the source tree https://github.com/ggml-org/llama.cpp/blob/master/tools/mtmd/README.md .
|
| 77 |
+
Use of the best available model (Q4_K_H) is recommended to maximize the accuracy of vision mode. To run it on a 12G VRAM
|
| 78 |
+
GPU use --ngl 32. Generation speed is still quite good with partial offload.
|
| 79 |
+
|
| 80 |
## Download the file from below:
|
| 81 |
| Link | Type | Size/e9 B | Notes |
|
| 82 |
|------|------|-----------|-------|
|