File size: 4,876 Bytes
3d1a8da d176c11 3d1a8da d176c11 3d1a8da d176c11 3d1a8da d176c11 3d1a8da d176c11 c2af3b2 3d1a8da d176c11 3d1a8da d176c11 3d1a8da d176c11 c2af3b2 3d1a8da d176c11 3d1a8da d176c11 78650ae d176c11 3d1a8da 78650ae 3d1a8da 78650ae d176c11 3d1a8da 78650ae |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 |
---
license: apache-2.0
language:
- en
base_model:
- ibm-granite/granite-docling-258M
pipeline_tag: image-text-to-text
tags:
- gguf
---
## ibm-granite-docling-258M-GGUF
This is the GGUF version of the [ibm-granite/granite-docling-258M](https://huggingface.co/ibm-granite/granite-docling-258M) model. It has been converted to GGUF format for optimized inference performance and compatibility with modern model runtimes.
### Model Information
---
- **Model Name**: granite-docling-258M
- **Base Model**: ibm-granite/granite-docling-258M
- **License**: Apache-2.0
- **Pipeline Tag**: image-text-to-text
- **Language**: English
- **Model Size**: 258M
- **Model Format**: GGUF
### Description
---
Granite Docling is a family of instruction-tuned models designed for document understanding tasks. These models are fine-tuned on a diverse set of tasks including document classification, information extraction, and question answering. The models are optimized for performance on document-centric tasks and can handle a variety of document formats and layouts.
### Usage
---
> **Important:** Make sure to include the `--special` flag so that special tokens are properly returned.
> Example:
>
> ```bash
> llama-server -hf danchev/ibm-granite-docling-258m-GGUF --special
> ```
<details>
<summary> Usage with docker </summary>
```bash
# Serve the model with Docker
docker run --rm -p 8080:8080 ghcr.io/danchev/llama.cpp:docling \
--server \
-hf danchev/ibm-granite-docling-258M-GGUF \
--host 0.0.0.0 \
--port 8080 \
--special # required for docling to work
```
</details>
<details>
<summary> Usage with llama.cpp </summary>
```bash
# Build llama.cpp from source
git clone [email protected]:ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build --config Release -j $(nproc)
# Serve the model
./build/bin/llama-server -hf danchev/ibm-granite-docling-258M-GGUF --special
```
</details>
#### Docling Example:
---
<details open>
<summary> 📄 PDF Conversion Using `docling` </summary>
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.12"
# dependencies = ["docling>=2.58.0", "requests>=2.32.5"]
# ///
import tempfile
import requests
from pydantic import AnyUrl
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import VlmPipelineOptions
from docling.datamodel.pipeline_options_vlm_model import ApiVlmOptions, ResponseFormat
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.pipeline.vlm_pipeline import VlmPipeline
pdf_url = "https://arxiv.org/pdf/1706.03762.pdf"
with tempfile.NamedTemporaryFile(suffix=".pdf") as f:
f.write(requests.get(pdf_url).content)
f.flush()
pipeline_options = VlmPipelineOptions(
enable_remote_services=True,
vlm_options=ApiVlmOptions(
url=AnyUrl("http://127.0.0.1:8080/v1/chat/completions"),
params={"model": "danchev/ibm-granite-docling-258m-GGUF"},
prompt="Convert this page to docling.",
temperature=0.0,
response_format=ResponseFormat.DOCTAGS,
),
)
doc_converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_options=pipeline_options, pipeline_cls=VlmPipeline
)
}
)
print(doc_converter.convert(f.name).document.export_to_markdown())
```
</details>
<details open>
<summary> 🖼️ Image Conversion Using `docling-core` </summary>
```python
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.12"
# dependencies = ["docling-core>=2.49.0", "Pillow>=11.3.0", "requests>=2.32.5"]
# ///
import base64
from io import BytesIO
from pathlib import Path
import requests
from docling_core.types.doc.base import ImageRefMode
from docling_core.types.doc.document import DoclingDocument, DocTagsDocument
from PIL import Image
img_url = "https://ibm.biz/docling-page-with-list"
img_bytes = requests.get(img_url).content
img_b64 = base64.b64encode(img_bytes).decode()
doctags = requests.post(
url="http://localhost:8080/v1/chat/completions",
json={
"model": "danchev/ibm-granite-docling-258M-GGUF",
"messages": [
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}},
{"type": "text", "text": "Convert this page to docling."},
],
}
],
}
).json()["choices"][0]["message"]["content"]
doc = DoclingDocument.load_from_doctags(
doctag_document=DocTagsDocument.from_doctags_and_image_pairs(
doctags=[doctags], images=[Image.open(BytesIO(img_bytes))]),
)
print(doc.export_to_markdown())
doc.save_as_html(Path("output.html"), image_mode=ImageRefMode.EMBEDDED)
```
</details>
|