Improve model card: Add pipeline tag, library name, abstract, and detailed overview

README.md

@@ -1,15 +1,49 @@
 ---
-license: mit
 base_model:
 - Qwen/Qwen2.5-VL-7B-Instruct
 language:
 - en
+license: mit
+library_name: transformers
+pipeline_tag: image-text-to-text
 ---
 
 # QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training
 
 This repository contains the model weights for QoQ-Med-VL-7B (Qwen Omni-Reasoning on Medical Questions), a multimodal clinical foundation model with reasoning capabilities.
 
+📚 [Paper](https://huggingface.co/papers/2506.00711) | 💻 [Code](https://github.com/DDVD233/QoQ_Med)
+
+## Abstract
+Clinical decision-making routinely demands reasoning over heterogeneous data, yet existing multimodal language models (MLLMs) remain largely vision-centric and fail to generalize across clinical specialties. To bridge this gap, we introduce QoQ-Med-7B/32B, the first open generalist clinical foundation model that jointly reasons across medical images, time-series signals, and text reports. QoQ-Med is trained with Domain-aware Relative Policy Optimization (DRPO), a novel reinforcement-learning objective that hierarchically scales normalized rewards according to domain rarity and modality difficulty, mitigating performance imbalance caused by skewed clinical data distributions. Trained on 2.61 million instruction tuning pairs spanning 9 clinical domains, we show that DRPO training boosts diagnostic performance by 43% in macro-F1 on average across all visual domains as compared to other critic-free training methods like GRPO. Furthermore, with QoQ-Med trained on intensive segmentation data, it is able to highlight salient regions related to the diagnosis, with an IoU 10x higher than open models while reaching the performance of OpenAI o4-mini. To foster reproducibility and downstream research, we release (i) the full model weights, (ii) the modular training pipeline, and (iii) all intermediate reasoning traces at this https URL .
+
+## Model Overview
+![QoQ-Med Model Overview](https://github.com/DDVD233/QoQ_Med/blob/main/images/model_training.jpeg?raw=true)
+
+QoQ-Med is the first open generalist clinical foundation model that jointly reasons across:
+- Medical images (2D/3D)
+- Time-series signals (ECG)
+- Text reports
+
+The model is trained with our novel Domain-aware Relative Policy Optimization (DRPO), a reinforcement learning objective that hierarchically scales normalized rewards according to domain rarity and modality difficulty, addressing performance imbalance in heterogeneous clinical data.
+
+## Key Features
+
+-   **Multimodal Integration**: Processes and reasons across 1D, 2D, and 3D clinical data
+-   **Domain-Aware Training**: DRPO balances learning across 9 clinical domains
+-   **Enhanced Interpretability**: Generates reasoning traces and highlights salient regions
+-   **State-of-the-Art Performance**: Outperforms existing open-source clinical MLLMs
+
+## Clinical Domains
+
+QoQ-Med spans multiple clinical specialties:
+-   Cardiology (ECG, Chest X-ray)
+-   Radiology (CT, MRI, Ultrasound)
+-   Dermatology
+-   Ophthalmology (Fundus)
+-   Pathology
+-   Mammography
+
 ## Model Weights
 
 | Model | Weights | Avg. Val Accuracy |
@@ -164,4 +198,8 @@ If you find the project useful, please cite the following papers:
   journal={arXiv preprint arXiv:2506.00711},
   year={2025}
 }
-```
+```
+
+## Important Note
+
+This model is intended for research purposes only. Before extensive real-world testing (like human trials), it is not suitable for clinical deployment. This is a research preview, not a product approved by federal agencies.