Improve model card: Add pipeline tag, library name, paper link, and detailed description
Browse filesThis PR enhances the model card for the CoReward-Qwen2.5-7B model by adding key metadata and enriching its content:
- `pipeline_tag: text-generation`: This tag is added to correctly categorize the model and improve its discoverability for users seeking text generation capabilities on the Hugging Face Hub.
- `library_name: transformers`: The model's `config.json` clearly indicates compatibility with the `transformers` library, as it defines `Qwen2ForCausalLM` as an architecture and specifies a `transformers_version`. Adding this metadata enables the automated "How to use" widget, providing users with predefined code snippets.
- **Paper Link:** A direct link to the paper, [Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models](https://huggingface.co/papers/2508.00410), is included for easy access to the research.
- **Model Description:** A detailed description based on the paper's abstract is added to provide a clearer understanding of the Co-rewarding framework and its contributions.
- **GitHub Repository:** The link to the official GitHub repository (`https://github.com/tmlr-group/Co-rewarding`) is made more prominent.
- **Framework Image:** The visual framework image from the GitHub README is embedded to quickly convey the method's overview.
- **Citation Update:** The BibTeX citation is updated to match the more accurate version provided in the project's GitHub README.
These changes significantly improve the model card's completeness and user-friendliness.
|
@@ -1,18 +1,37 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
-
## CoReward-Qwen2.5-7B
|
| 5 |
|
| 6 |
-
|
| 7 |
|
| 8 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
|
| 10 |
## Citation
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
|
|
|
|
|
|
| 17 |
}
|
| 18 |
```
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
pipeline_tag: text-generation
|
| 4 |
+
library_name: transformers
|
| 5 |
---
|
|
|
|
| 6 |
|
| 7 |
+
# Co-rewarding: Qwen2.5-7B Model
|
| 8 |
|
| 9 |
+
This is the Qwen2.5-7B model trained by the Co-rewarding method using the MATH training set, as presented in the paper [Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models](https://huggingface.co/papers/2508.00410).
|
| 10 |
+
|
| 11 |
+
<p align="center">
|
| 12 |
+
<img src="https://github.com/tmlr-group/Co-rewarding/raw/main/figs/Method.png" alt="Co-rewarding Framework">
|
| 13 |
+
</p>
|
| 14 |
+
|
| 15 |
+
## Model Description
|
| 16 |
+
|
| 17 |
+
While reinforcement learning with verifiable rewards (RLVR) is effective to improve the reasoning ability of large language models (LLMs), its reliance on human-annotated labels leads to the scaling up dilemma, especially for complex tasks. Recent self-rewarding methods investigate a label-free alternative to unlock the reasoning capabilities of LLMs, yet they frequently encounter the non-negligible training collapse issue, as the single-view supervision signal easily forms the self-consistent illusion, yielding the reward hacking.
|
| 18 |
+
|
| 19 |
+
**Co-rewarding** is a novel self-supervised RL framework that improves training stability by seeking complementary supervision from another views. Specifically, Co-rewarding is instantiated in two ways:
|
| 20 |
+
1. **Co-rewarding-I**: A data-side instantiation that derives reward signals from contrastive agreement across semantically analogous questions.
|
| 21 |
+
2. **Co-rewarding-II**: A model-side instantiation that maintains a slowly-updated reference teacher with pseudo labels to realize self-distillation.
|
| 22 |
+
|
| 23 |
+
Intuitively, such instantiations introduce different levels of discrepancy to increase the difficulty of training collapse on trivial reasoning solutions. Empirically, Co-rewarding exhibits stable training across various setups, and outperforms other self-rewarding baselines by $+3.31\%$ improvements on average on multiple mathematical reasoning benchmarks, especially by $+7.49\%$ on Llama-3.2-3B-Instruct. Notably, Co-rewarding reaches or even surpasses RLVR with ground-truth (GT) label in several cases, such as a Pass@$1$ of $94.01\%$ on GSM8K with Qwen3-8B-Base.
|
| 24 |
+
|
| 25 |
+
For more details about the Co-rewarding method, including code and training scripts, please refer to the official [Github Repository](https://github.com/tmlr-group/Co-rewarding).
|
| 26 |
|
| 27 |
## Citation
|
| 28 |
+
|
| 29 |
+
If you use our datasets or models, please cite our paper:
|
| 30 |
+
```bibtex
|
| 31 |
+
@article{zhang2025co,
|
| 32 |
+
title={Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models},
|
| 33 |
+
author={Zhang, Zizhuo and Zhu, Jianing and Ge, Xinmu and Zhao, Zihua and Zhou, Zhanke and Li, Xuan and Feng, Xiao and Yao, Jiangchao and Han, Bo},
|
| 34 |
+
journal={arXiv preprint arXiv:2508.00410},
|
| 35 |
+
year={2025}
|
| 36 |
}
|
| 37 |
```
|