Image-Text-to-Text
Transformers
Safetensors
multilingual
deepseek_vl_v2
feature-extraction
multimodal
ocr
vision-language
document-intelligence
conversational
custom_code
Eval Results (legacy)
Eval Results
🇪🇺 Region: EU
Instructions to use jinaai/jina-ocr-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jinaai/jina-ocr-v1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="jinaai/jina-ocr-v1", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("jinaai/jina-ocr-v1", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use jinaai/jina-ocr-v1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jinaai/jina-ocr-v1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jinaai/jina-ocr-v1", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/jinaai/jina-ocr-v1
- SGLang
How to use jinaai/jina-ocr-v1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "jinaai/jina-ocr-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jinaai/jina-ocr-v1", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "jinaai/jina-ocr-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jinaai/jina-ocr-v1", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use jinaai/jina-ocr-v1 with Docker Model Runner:
docker model run hf.co/jinaai/jina-ocr-v1
fix: remove loguru
Browse files- README.md +23 -4
- deepencoder.py +0 -4
- deepseek_ocr_mtp.py +1 -2
- example.py +1 -7
- modeling_deepseekocr.py +2 -4
- processing_deepseek_ocr.py +1 -2
README.md
CHANGED
|
@@ -10,8 +10,6 @@ tags:
|
|
| 10 |
- ocr
|
| 11 |
- vision-language
|
| 12 |
- document-intelligence
|
| 13 |
-
base_model:
|
| 14 |
-
- deepseek-ai/DeepSeek-OCR
|
| 15 |
inference: false
|
| 16 |
model-index:
|
| 17 |
- name: Jina-OCR-v1
|
|
@@ -54,6 +52,9 @@ This repository ships the model weights **and** the custom code needed to load t
|
|
| 54 |
| ------------ | ----------------------------------------------------------- |
|
| 55 |
| Backbone | DeepSeek-OCR (DeepEncoder + 3B MoE, ~570M active) |
|
| 56 |
| Acceleration | FastMTP, one dense draft head, K=3 (vLLM only) |
|
|
|
|
|
|
|
|
|
|
| 57 |
| Local run | [`example.py`](example.py) — Transformers or vLLM |
|
| 58 |
| Hosted | OpenAI-compatible `https://api.jina.ai/v1/chat/completions` |
|
| 59 |
|
|
@@ -62,6 +63,24 @@ python example.py --backend transformers --image document.png
|
|
| 62 |
python example.py --backend vllm --image document.png
|
| 63 |
```
|
| 64 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
## Prompt
|
| 66 |
|
| 67 |
Recommended default:
|
|
@@ -193,13 +212,13 @@ Chat content types `image`, `image_pil`, and `image_url` all insert `<image>`. D
|
|
| 193 |
|
| 194 |
```bibtex
|
| 195 |
@misc{garcía2026jinaocr,
|
| 196 |
-
title={Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards},
|
| 197 |
author={Alejandro Barón García and Feng Wang and Emilia Garcia Casademont and Han Xiao},
|
| 198 |
year={2026},
|
| 199 |
eprint={2609.03181},
|
| 200 |
archivePrefix={arXiv},
|
| 201 |
primaryClass={cs.CL},
|
| 202 |
-
url={https://arxiv.org/abs/2609.03181},
|
| 203 |
}
|
| 204 |
```
|
| 205 |
|
|
|
|
| 10 |
- ocr
|
| 11 |
- vision-language
|
| 12 |
- document-intelligence
|
|
|
|
|
|
|
| 13 |
inference: false
|
| 14 |
model-index:
|
| 15 |
- name: Jina-OCR-v1
|
|
|
|
| 52 |
| ------------ | ----------------------------------------------------------- |
|
| 53 |
| Backbone | DeepSeek-OCR (DeepEncoder + 3B MoE, ~570M active) |
|
| 54 |
| Acceleration | FastMTP, one dense draft head, K=3 (vLLM only) |
|
| 55 |
+
| OmniDocBench v1.6 | 91.14 overall |
|
| 56 |
+
| olmOCR-Bench | 83.4 overall (+7.4 vs DeepSeek-OCR) |
|
| 57 |
+
| Throughput | 2.57 pages/s on olmOCR-Bench (A100, concurrency 32) |
|
| 58 |
| Local run | [`example.py`](example.py) — Transformers or vLLM |
|
| 59 |
| Hosted | OpenAI-compatible `https://api.jina.ai/v1/chat/completions` |
|
| 60 |
|
|
|
|
| 63 |
python example.py --backend vllm --image document.png
|
| 64 |
```
|
| 65 |
|
| 66 |
+
## Performance
|
| 67 |
+
|
| 68 |
+
At the default dynamic-resolution setting, `jina-ocr-v1` scores **91.14** on OmniDocBench v1.6 and **83.4** on olmOCR-Bench, and has the highest page throughput of the fourteen systems we measured: **2.57 pages/s**. Post-training adds 7.4 points on olmOCR-Bench over the DeepSeek-OCR backbone. On an NVIDIA L4, FastMTP nearly doubles decoding speed over greedy autoregressive decoding. Scores below use the [eval prompt](#prompt).
|
| 69 |
+
|
| 70 |
+
<p align="center">
|
| 71 |
+
<img src="https://cms.jina.ai/content/images/size/w1600/2026/09/jina-ocr-v1-overview.png" alt="jina-ocr-v1 on three deployment-cost axes: pixels per visual token, page throughput, and accuracy versus active parameters">
|
| 72 |
+
</p>
|
| 73 |
+
|
| 74 |
+
Three axes that determine deployment cost. **(a)** Pixels per visual token, log scale. DeepEncoder maps a 1024×1024 view from 4,096 patches to 256 tokens (3,887 pixels per visual token vs 783–1,022 for 28–32 px patch encoders). **(b)** Page throughput on olmOCR-Bench, one A100, concurrency 32. **(c)** Benchmark overall vs active parameters (log scale); the solid line joins Pareto-optimal systems. `jina-ocr-v1` sits on both frontiers at 570M active parameters.
|
| 75 |
+
|
| 76 |
+
<p align="center">
|
| 77 |
+
<img src="https://cms.jina.ai/content/images/size/w1600/2026/09/jina-ocr-v1-efficiency.png" alt="olmOCR-Bench efficiency: output tokens per second, tokens per page, and pages per second">
|
| 78 |
+
</p>
|
| 79 |
+
|
| 80 |
+
The same fourteen systems on olmOCR-Bench (one A100, concurrency 32), ranked by **(a)** output tokens/s, **(b)** output tokens/page, and **(c)** pages/s. Surya OCR 2 leads on tokens/s (3,760) but emits 3,568 tokens/page and finishes 1.05 pages/s. `jina-ocr-v1` combines 2,792 tokens/s with 1,085 tokens/page and reaches 2.57 pages/s — the shortest outputs of any system scoring above 83.
|
| 81 |
+
|
| 82 |
+
Per-category breakdowns on olmOCR-Bench and OmniDocBench v1.6, FastMTP measurements on L4, and the full comparison set are in the [technical report](https://arxiv.org/abs/2609.03181) and the [blog](https://jina.ai/news/jina-ocr-v1-faster-document-parsing-on-low-budget-gpus/).
|
| 83 |
+
|
| 84 |
## Prompt
|
| 85 |
|
| 86 |
Recommended default:
|
|
|
|
| 212 |
|
| 213 |
```bibtex
|
| 214 |
@misc{garcía2026jinaocr,
|
| 215 |
+
title={Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards},
|
| 216 |
author={Alejandro Barón García and Feng Wang and Emilia Garcia Casademont and Han Xiao},
|
| 217 |
year={2026},
|
| 218 |
eprint={2609.03181},
|
| 219 |
archivePrefix={arXiv},
|
| 220 |
primaryClass={cs.CL},
|
| 221 |
+
url={https://arxiv.org/abs/2609.03181},
|
| 222 |
}
|
| 223 |
```
|
| 224 |
|
deepencoder.py
CHANGED
|
@@ -431,9 +431,6 @@ class VitModel(nn.Module):
|
|
| 431 |
self.transformer = NoTPTransformer(cfg=cfg)
|
| 432 |
|
| 433 |
if cfg.get("fp32norm", False):
|
| 434 |
-
from loguru import logger
|
| 435 |
-
|
| 436 |
-
logger.info("Load fp32 layernorm for ViT.")
|
| 437 |
self.pre_layrnorm = LayerNormfp32(
|
| 438 |
cfg.hidden_size,
|
| 439 |
eps=cfg.get("pre_layernorm_epsilon", 1e-5),
|
|
@@ -1043,5 +1040,4 @@ def _build_sam(
|
|
| 1043 |
image_encoder.load_state_dict(
|
| 1044 |
{k[30:]: v for k, v in state_dict.items() if "vision_tower_high" in k}, strict=True
|
| 1045 |
)
|
| 1046 |
-
print(checkpoint)
|
| 1047 |
return image_encoder
|
|
|
|
| 431 |
self.transformer = NoTPTransformer(cfg=cfg)
|
| 432 |
|
| 433 |
if cfg.get("fp32norm", False):
|
|
|
|
|
|
|
|
|
|
| 434 |
self.pre_layrnorm = LayerNormfp32(
|
| 435 |
cfg.hidden_size,
|
| 436 |
eps=cfg.get("pre_layernorm_epsilon", 1e-5),
|
|
|
|
| 1040 |
image_encoder.load_state_dict(
|
| 1041 |
{k[30:]: v for k, v in state_dict.items() if "vision_tower_high" in k}, strict=True
|
| 1042 |
)
|
|
|
|
| 1043 |
return image_encoder
|
deepseek_ocr_mtp.py
CHANGED
|
@@ -491,8 +491,7 @@ def log_spec_stats(llm) -> None:
|
|
| 491 |
|
| 492 |
|
| 493 |
DEFAULT_OCR_PROMPT = (
|
| 494 |
-
"Transcribe the provided document image into a clean Markdown format, "
|
| 495 |
-
"preserving the natural reading order."
|
| 496 |
)
|
| 497 |
|
| 498 |
|
|
|
|
| 491 |
|
| 492 |
|
| 493 |
DEFAULT_OCR_PROMPT = (
|
| 494 |
+
"Transcribe the provided document image into a clean Markdown format, " "preserving the natural reading order."
|
|
|
|
| 495 |
)
|
| 496 |
|
| 497 |
|
example.py
CHANGED
|
@@ -51,13 +51,7 @@ def run_vllm(
|
|
| 51 |
max_new_tokens: int,
|
| 52 |
num_speculative_tokens: int,
|
| 53 |
) -> str:
|
| 54 |
-
from deepseek_ocr_mtp import
|
| 55 |
-
DEFAULT_OCR_PROMPT,
|
| 56 |
-
log_spec_stats,
|
| 57 |
-
register,
|
| 58 |
-
vllm_llm_kwargs,
|
| 59 |
-
vllm_sampling_params,
|
| 60 |
-
)
|
| 61 |
from vllm import LLM
|
| 62 |
|
| 63 |
register()
|
|
|
|
| 51 |
max_new_tokens: int,
|
| 52 |
num_speculative_tokens: int,
|
| 53 |
) -> str:
|
| 54 |
+
from deepseek_ocr_mtp import DEFAULT_OCR_PROMPT, log_spec_stats, register, vllm_llm_kwargs, vllm_sampling_params
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
from vllm import LLM
|
| 56 |
|
| 57 |
register()
|
modeling_deepseekocr.py
CHANGED
|
@@ -11,8 +11,8 @@ from transformers import LogitsProcessorList, TextStreamer
|
|
| 11 |
from transformers.modeling_outputs import BaseModelOutputWithPast, MoeCausalLMOutputWithPast
|
| 12 |
from transformers.utils import logging
|
| 13 |
|
| 14 |
-
from .deepencoder import MlpProjector, _AttrDict, build_clip_l, build_sam_vit_b
|
| 15 |
from .configuration_deepseek_v2 import DeepseekV2Config
|
|
|
|
| 16 |
from .modeling_deepseekv2 import (
|
| 17 |
DeepseekV2ForCausalLM,
|
| 18 |
DeepseekV2Model,
|
|
@@ -444,9 +444,7 @@ class DeepseekOCRForCausalLM(DeepseekV2ForCausalLM):
|
|
| 444 |
processors = LogitsProcessorList(processors)
|
| 445 |
already = any(isinstance(p, SlidingWindowNoRepeatNgramProcessor) for p in processors)
|
| 446 |
if ngram_size and window_size and not already:
|
| 447 |
-
processors.append(
|
| 448 |
-
SlidingWindowNoRepeatNgramProcessor(ngram_size, window_size, whitelist_token_ids)
|
| 449 |
-
)
|
| 450 |
if processors:
|
| 451 |
kwargs["logits_processor"] = processors
|
| 452 |
return super().generate(*args, **kwargs)
|
|
|
|
| 11 |
from transformers.modeling_outputs import BaseModelOutputWithPast, MoeCausalLMOutputWithPast
|
| 12 |
from transformers.utils import logging
|
| 13 |
|
|
|
|
| 14 |
from .configuration_deepseek_v2 import DeepseekV2Config
|
| 15 |
+
from .deepencoder import MlpProjector, _AttrDict, build_clip_l, build_sam_vit_b
|
| 16 |
from .modeling_deepseekv2 import (
|
| 17 |
DeepseekV2ForCausalLM,
|
| 18 |
DeepseekV2Model,
|
|
|
|
| 444 |
processors = LogitsProcessorList(processors)
|
| 445 |
already = any(isinstance(p, SlidingWindowNoRepeatNgramProcessor) for p in processors)
|
| 446 |
if ngram_size and window_size and not already:
|
| 447 |
+
processors.append(SlidingWindowNoRepeatNgramProcessor(ngram_size, window_size, whitelist_token_ids))
|
|
|
|
|
|
|
| 448 |
if processors:
|
| 449 |
kwargs["logits_processor"] = processors
|
| 450 |
return super().generate(*args, **kwargs)
|
processing_deepseek_ocr.py
CHANGED
|
@@ -27,8 +27,7 @@ DS_OCR_TILE_SIZE = 640
|
|
| 27 |
DS_OCR_BASE_SIZE = 1024
|
| 28 |
|
| 29 |
DEFAULT_OCR_PROMPT = (
|
| 30 |
-
"Transcribe the provided document image into a clean Markdown format, "
|
| 31 |
-
"preserving the natural reading order."
|
| 32 |
)
|
| 33 |
|
| 34 |
DS_OCR_DEFAULT_CHAT_TEMPLATE = dedent(
|
|
|
|
| 27 |
DS_OCR_BASE_SIZE = 1024
|
| 28 |
|
| 29 |
DEFAULT_OCR_PROMPT = (
|
| 30 |
+
"Transcribe the provided document image into a clean Markdown format, " "preserving the natural reading order."
|
|
|
|
| 31 |
)
|
| 32 |
|
| 33 |
DS_OCR_DEFAULT_CHAT_TEMPLATE = dedent(
|