Improve model card: Add pipeline tag, library, abstract, and comprehensive usage

This PR significantly enhances the model card for UniLIP-3B by:

- Adding the `pipeline_tag: any-to-any`, accurately reflecting the model's unified multimodal capabilities in understanding, generation, and editing.
- Specifying `library_name: transformers` as the model is compatible with the 🤗 Transformers library (evidence from `config.json` showing `transformers_version`). This enables automated code snippets for easy usage on the Hub.
- Updating the paper link to point directly to the Hugging Face papers page: [UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing](https://huggingface.co/papers/2507.23278).
- Including the full paper abstract, providing essential context about the model's design and contributions.
- Adding detailed "Main Results" tables to showcase the model's performance across various tasks.
- Integrating a comprehensive "Quick Start" section, including installation instructions and "Simple Inference" code snippets for image generation, understanding, and omni-potent tasks, directly from the official GitHub repository. The `model_path` in the code has been updated to refer to `kanashi6/UniLIP-3B` for direct usability.
- Including "Acknowledgements" and "Citation" sections for proper attribution.

These changes will make the model more discoverable, informative, and user-friendly on the Hugging Face Hub.

README.md

@@ -1,5 +1,6 @@
 ---
-license: apache-2.0
+base_model:
+- OpenGVLab/InternVL3-2B
 datasets:
 - BLIP3o/BLIP3o-Pretrain-Long-Caption
 - BLIP3o/BLIP3o-Pretrain-Short-Caption
@@ -7,15 +8,190 @@ datasets:
 - UCSC-VLAA/GPT-Image-Edit-1.5M
 - BLIP3o/BLIP3o-60k
 - FreedomIntelligence/ShareGPT-4o-Image
-base_model:
-- OpenGVLab/InternVL3-2B
+license: apache-2.0
+pipeline_tag: any-to-any
+library_name: transformers
 ---
-This repository contains the model (3B version) presented in the paper UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing.
+
+# UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing
+
+This repository contains the model (3B version) presented in the paper [UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing](https://huggingface.co/papers/2507.23278).
 
 UniLIP proposes a unified, CLIP-based encoder featuring both rich semantics and fine-grained image details. Through a **two-stage and self-distillation training** for reconstruction, we empower CLIP to achieve excellent reconstruction results **without compromising its original understanding abilities**. Leveraging this powerful unified representation, UniLIP excels across understanding, generation, and editing tasks.
 
 For more details, please refer to the original paper and the GitHub repository:
 
-Paper: https://www.arxiv.org/abs/2507.23278
+Paper: [UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing](https://huggingface.co/papers/2507.23278)
+
+GitHub: https://github.com/nnnth/UniLIP
+
+<div align="center">
+  <img src="https://github.com/nnnth/UniLIP/raw/main/assets/img1.png" width="800"/>
+</div>
+
+## Abstract
+
+In this paper, we propose UniLIP, a unified framework that adapts CLIP for multimodal understanding, generation and editing. Although CLIP excels at understanding, it lacks reconstruction abilities required to be a unified visual encoder. However, previous CLIP-based unified methods fail to balance understanding and reconstruction, leading to semantic degradation or inconsistent reconstructions. In contrast, we introduce a novel two-stage training scheme with a self-distillation strategy that progressively endows CLIP with high-fidelity reconstruction abilities while preserving its original comprehension performance. For enhanced reasoning and consistency in generation and editing, we further develop a dual-condition architecture built upon the MetaQuery framework. Our architecture jointly utilizes multimodal hidden states for rich contextual details and learnable query embeddings to harness the powerful reasoning abilities of Multimodal Large Language Models (MLLMs). Leveraging advanced image representation and architectural design, UniLIP demonstrates superior instruction following and edit fidelity. With only 1B and 3B parameters, UniLIP can outperform larger unified models such as BAGEL (7B) and Uniworld-V1 (12B), achieving state-of-the-art performance of 0.90 on GenEval, 0.63 on WISE, and 3.94 on ImgEdit. These results demonstrate that UniLIP successfully expands the application of CLIP, establishing its continuous features to not only serve as the optimal choice for understanding tasks but also achieve highly competitive performance in generation and editing tasks. Code and models are available at this https URL .
+
+## Main Results
+
+### Image Reconstruction on ImageNet val
+
+| Model | Res. | ratio | rFID ↓ | PSNR↑ | SSIM↑ |
+| :--- | :--- | :--- | :--- | :--- | :--- |
+| VILA-U | 256 | 16 | 1.80 | - | - |
+| Tokenflow | 256 | 16 | 1.37 | 21.41 | 0.687 |
+| DualViTok | 256 | 16 | 1.37 | 22.53 | 0.741 |
+| **UniLIP** | 256 | 32 | **0.79** | **22.99** | **0.747** |
+| Emu2 | 448 | 14 | 3.27 | 13.49 | 0.423 |
+| **UniLIP** | 448 | 32 | **0.31** | **24.62** | **0.788** |
+ 
+
+### Image Understanding
+| Model | # LLM Params | MME-P | MMB | MMMU | MM-Vet | SEED | AI2D | MMVP |
+| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
+| InternVL3-1B | 1B | 1492 | 72.6 | 43.4 | 59.5 | 71.1 | 69.4 | 67.3 |
+| InternVL3-2B | 1.8B | 1633 | 80.6 | 48.2 | 62.2 | 75.0 | 78.5 | 72.7 |
+| BAGEL-3B | 3B | 1610 | 79.2 | 43.2 | 48.2 | - | - | 54.7 |
+| BLIP3o-4B | 4B | 1528 | 78.6 | 46.6 | 60.1 | 73.8 | - | - |
+| TokLIP-7B | 7B | 1410 | - | 42.1 | - | 65.2 | - | - |
+| Tar-7B | 7B | 1571 | 74.4 | 39.0 | | 73.0 | - | - |
+| **UniLIP-1B** | 1B | 1499 | 72.6 | 43.3 | 59.4 | 71.0 | 70.7 | 68.7 |
+| **UniLIP-3B** | 2B | **1636** | **80.7** | **48.7** | **62.2** | **75.0** | **78.6** | **73.0** |
+
+
+
+### Image Generation and Editing
+| Model | # Params | GenEval | WISE  | ImgEdit |
+| :--- | :--- | :--- | :--- | :--- |
+| BAGEL | 7B+7B | 0.82 | 0.52 | 3.20 |
+| BLIP3o-4B | 3B+1.4B | 0.81 | 0.50 | - |
+| UniWorld-V1 | 7B+12B | - | - | 3.26 |
+| **UniLIP-1B** | 1B+0.6B | 0.88 | 0.56 | 3.81 |
+| **UniLIP-3B** | 2B+1.6B | **0.90** | **0.63** | **3.94** |
+
+## 🛠️ Quick Start
+
+### Installation
+
+```shell
+conda create -n UniLIP python=3.11
+conda activate UniLIP
+pip install torch==2.6.0+cu118 torchvision==0.21.0+cu118 --index-url https://download.pytorch.org/whl/cu118
+pip install -r requirements.txt
+pip install -e .
+```
+
+### Simple Inference
+
+The simplest code for UniLiP inference, covering generation, understanding, and omni-potent tasks:
+
+```python
+from inference_solver import FlexARInferenceSolver
+from PIL import Image
+
+# ******************** Image Generation ********************
+print("--- Image Generation Example ---")
+inference_solver = FlexARInferenceSolver(
+    model_path="kanashi6/UniLIP-3B", # Using the model from this repository
+    precision="bf16",
+    target_size=768,
+)
+
+q1 = f"Generate an image of 768x768 according to the following prompt:
+" \
+     f"Image of a dog playing water, and a waterfall is in the background."
+
+# generated: tuple of (generated response, list of generated images)
+generated = inference_solver.generate(
+    images=[],
+    qas=[[q1, None]],
+    max_gen_len=8192,
+    temperature=1.0,
+    logits_processor=inference_solver.create_logits_processor(cfg=4.0, image_top_k=2000),
+)
+
+a1, new_image = generated[0], generated[1][0]
+print(f"Generated text: {a1}")
+# new_image.save("generated_dog_image.png") # Uncomment to save the generated image
+
+
+# ******************* Image Understanding ******************
+print("
+--- Image Understanding Example ---")
+inference_solver_understanding = FlexARInferenceSolver(
+    model_path="kanashi6/UniLIP-3B", # Using the model from this repository
+    precision="bf16",
+    target_size=512,
+)
+
+# "<|image|>" symbol will be replaced with sequence of image tokens before fed to LLM
+q2 = "Describe the image in detail. <|image|>"
+
+# Assuming 'image.png' exists in the current directory for demonstration
+# Example: images = [Image.open("path/to/your/image.png")]
+images_for_understanding = [Image.new('RGB', (512, 512), color = 'white')] # Placeholder if no file found
+qas_for_understanding = [[q2, None]]
+
+# `len(images)` should be equal to the number of appearance of "<|image|>" in qas
+generated_understanding = inference_solver_understanding.generate(
+    images=images_for_understanding,
+    qas=qas_for_understanding,
+    max_gen_len=8192,
+    temperature=1.0,
+    logits_processor=inference_solver_understanding.create_logits_processor(cfg=4.0, image_top_k=2000),
+)
+
+a2 = generated_understanding[0]
+print(f"Generated description: {a2}")
+# generated[1], namely the list of newly generated images, should typically be empty in this case.
+
+
+# ********************* Omni-Potent Example (Depth Estimation) *********************
+print("
+--- Omni-Potent Example (Depth Estimation) ---")
+inference_solver_omni = FlexARInferenceSolver(
+    model_path="kanashi6/UniLIP-3B", # Using the model from this repository
+    precision="bf16",
+    target_size=768,
+)
+
+# Example: Depth Estimation
+# For more instructions, see demos/demo_image2image.py
+q3 = "Depth estimation. <|image|>"
+# Assuming 'image.png' exists in the current directory for demonstration
+# Example: images = [Image.open("path/to/your/image.png")]
+images_for_omni = [Image.new('RGB', (768, 768), color = 'gray')] # Placeholder if no file found
+qas_for_omni = [[q3, None]]
+
+generated_omni = inference_solver_omni.generate(
+    images=images_for_omni,
+    qas=qas_for_omni,
+    max_gen_len=8192,
+    temperature=1.0,
+    logits_processor=inference_solver_omni.create_logits_processor(cfg=1.0, image_top_k=200),
+)
+
+a3 = generated_omni[0]
+new_image_omni = generated_omni[1][0]
+print(f"Generated text (depth estimation): {a3}")
+# new_image_omni.save("generated_depth_map.png") # Uncomment to save the generated depth map
+```
+
+## 👍 Acknowledgement
+*   [TiTok](https://github.com/bytedance/1d-tokenizer) We implement reconstruction training following TiTok.
+*   [BLIP3-o](https://github.com/JiuhaiChen/BLIP3o/tree/main) Thanks to BLIP3-o for providing the generation data and training code.
+*   [InternVL](https://github.com/OpenGVLab/InternVL) We use InternVL3 as the MLLM pretraining.
+*   [SANA](https://github.com/NVlabs/Sana) We use SANA as the DiT pretraining.
+*   [DC-AE](https://github.com/mit-han-lab/efficientvit) We use the pixel decoder from DC-AE.
 
-GitHub: https://github.com/nnnth/UniLIP
+## 📘 Citation
+Please consider citing our work as follows if it is helpful.
+```bibtex
+@article{tang2025unilip,
+  title={UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing},
+  author={Tang, Hao and Xie, Chenwei and Bao, Xiaoyi and Weng, Tingyu and Li, Pandeng and Zheng, Yun and Wang, Liwei},
+  journal={arXiv preprint arXiv:2507.23278},
+  year={2025}
+}
+```