Video-ORA-9B

Paper · Project page · Code · Evaluation data

Video-ORA-9B is the 9B model released with Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs. It is fine-tuned from Qwen3.5-9B with OraRL across seven video and multimodal task families:

• temporal grounding
• visual tracking
• image and video segmentation
• spatial grounding
• spatial-temporal grounding
• video question answering
• spatial intelligence

The model is evaluated with direct, task-native answers rather than generated chain-of-thought.

Model summary

Property	Value
Architecture	Qwen3_5ForConditionalGeneration
Parameters	9B
Weight dtype	BF16
Native context length	262,144 tokens
Base model	Qwen/Qwen3.5-9B
Post-training	OraRL annotation-augmented on-policy reinforcement learning
Tested serving	Transformers 5.5.4 and vLLM 0.19.1

Results

Dataset-level comparison

Video-ORA-9B leads the matched seven-family comparison without chain-of-thought decoding. Best and second-best values are highlighted per row; † denotes an original-report value whose frame, prompt, split, or decoding settings may differ. Averages require complete family coverage.

See the OraRL repository, project page, and paper for complete benchmark protocols and source attribution.

Quick start

vLLM serving

pip install "vllm==0.19.1" openai

vllm serve OraRL/Video-ORA-9B \
  --served-model-name Video-ORA-9B \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --media-io-kwargs '{"video": {"num_frames": -1}}' \
  --limit-mm-per-prompt '{"image": 1, "video": 1}'

Reduce --max-model-len if KV-cache memory is limited. Increase --tensor-parallel-size for multi-GPU serving. Model-weight loading occupies approximately 17.6 GiB in the tested BF16 vLLM environment; this is not a full peak-memory measurement.

Send an OpenAI-compatible video request:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="Video-ORA-9B",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "video_url",
                    "video_url": {
                        "url": "https://orarl.github.io/assets/orarl-teaser.mp4"
                    },
                },
                {
                    "type": "text",
                    "text": "Describe the video and answer the question directly.",
                },
            ],
        }
    ],
    max_tokens=128,
    temperature=0.0,
    extra_body={
        "top_k": 20,
        "chat_template_kwargs": {"enable_thinking": False},
        "mm_processor_kwargs": {"fps": 2, "do_sample_frames": True},
    },
)
print(response.choices[0].message.content)

Replace the demo URL with your own accessible video URL. For local files, launch vLLM with an appropriate --allowed-local-media-path.

Transformers server

Qwen3.5 requires a recent Transformers version:

pip install "transformers[serving] @ git+https://github.com/huggingface/transformers.git@main"
pip install accelerate torchvision pillow

transformers serve \
  --force-model OraRL/Video-ORA-9B \
  --port 8000 \
  --continuous-batching

The checkpoint also contains the tokenizer, processor configuration, generation configuration, and chat template required by compatible Transformers and vLLM releases. Task-specific prompts and output schemas are documented on the project page.

Intended use

Video-ORA-9B is intended for research on structured image/video perception, benchmark evaluation, and task-specific adaptation. Use direct answer prompts with enable_thinking=False to match the reported evaluation protocol.

Out-of-scope uses include safety-critical decisions, identity inference, surveillance deployment, or use that violates the licenses or consent requirements of upstream media.

Training data

Training uses public training splits from the task families described in the paper. Evaluation identities, questions, and media anchors are excluded during mixture construction. Dataset and media licenses remain governed by their original sources; no training or benchmark media are distributed with this checkpoint.

Limitations

Video-ORA-9B is a research checkpoint optimized for structured video and spatial-understanding tasks. It may produce malformed task-specific outputs, hallucinate visual details, or inherit limitations and biases from its base model and training data. It has not been validated for safety-critical or high-stakes use.

License

The checkpoint is released under the Apache License 2.0. It is derived from Qwen3.5-9B, which is also distributed under Apache 2.0.

Citation

@article{li2026orarl,
  title   = {Annotations as Rollouts: Efficient and Scalable
             Reinforcement Learning for Video MLLMs},
  author  = {Li, Yunheng and Mu, Guohong and Li, Hao and
             Qian, Shengsheng and Zhang, Dingwen and Hou, Qibin
             and Cheng, Ming-Ming},
  journal = {arXiv preprint arXiv:2608.20492},
  year    = {2026},
  url     = {https://arxiv.org/abs/2608.20492}
}