nielsr HF Staff commited on
Commit
cc75233
·
verified ·
1 Parent(s): 1cc4a86

Improve model card: Add pipeline tag, library name, paper link, and detailed description

Browse files

This PR enhances the model card for the CoReward-Qwen2.5-7B model by adding key metadata and enriching its content:

- `pipeline_tag: text-generation`: This tag is added to correctly categorize the model and improve its discoverability for users seeking text generation capabilities on the Hugging Face Hub.
- `library_name: transformers`: The model's `config.json` clearly indicates compatibility with the `transformers` library, as it defines `Qwen2ForCausalLM` as an architecture and specifies a `transformers_version`. Adding this metadata enables the automated "How to use" widget, providing users with predefined code snippets.
- **Paper Link:** A direct link to the paper, [Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models](https://huggingface.co/papers/2508.00410), is included for easy access to the research.
- **Model Description:** A detailed description based on the paper's abstract is added to provide a clearer understanding of the Co-rewarding framework and its contributions.
- **GitHub Repository:** The link to the official GitHub repository (`https://github.com/tmlr-group/Co-rewarding`) is made more prominent.
- **Framework Image:** The visual framework image from the GitHub README is embedded to quickly convey the method's overview.
- **Citation Update:** The BibTeX citation is updated to match the more accurate version provided in the project's GitHub README.

These changes significantly improve the model card's completeness and user-friendliness.

Files changed (1) hide show
  1. README.md +28 -9
README.md CHANGED
@@ -1,18 +1,37 @@
1
  ---
2
  license: mit
 
 
3
  ---
4
- ## CoReward-Qwen2.5-7B
5
 
6
- This is the Qwen2.5-7B model trained by Co-Reward method using MATH training set.
7
 
8
- If you are interested in Co-Reward, you can find more details on our Github Repo [https://github.com/tmlr-group/Co-Reward].
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
 
10
  ## Citation
11
- ```
12
- @article{zhang2025coreward,
13
- title={Co-Reward: Self-supervised Reinforcement Learning for Large Language Model Reasoning via Contrastive Agreement},
14
- author={Zizhuo Zhang and Jianing Zhu and Xinmu Ge and Zihua Zhao and Zhanke Zhou and Xuan Li and Xiao Feng and Jiangchao Yao and Bo Han},
15
- journal={arXiv preprint arXiv:2508.00410}
16
- year={2025},
 
 
17
  }
18
  ```
 
1
  ---
2
  license: mit
3
+ pipeline_tag: text-generation
4
+ library_name: transformers
5
  ---
 
6
 
7
+ # Co-rewarding: Qwen2.5-7B Model
8
 
9
+ This is the Qwen2.5-7B model trained by the Co-rewarding method using the MATH training set, as presented in the paper [Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models](https://huggingface.co/papers/2508.00410).
10
+
11
+ <p align="center">
12
+ <img src="https://github.com/tmlr-group/Co-rewarding/raw/main/figs/Method.png" alt="Co-rewarding Framework">
13
+ </p>
14
+
15
+ ## Model Description
16
+
17
+ While reinforcement learning with verifiable rewards (RLVR) is effective to improve the reasoning ability of large language models (LLMs), its reliance on human-annotated labels leads to the scaling up dilemma, especially for complex tasks. Recent self-rewarding methods investigate a label-free alternative to unlock the reasoning capabilities of LLMs, yet they frequently encounter the non-negligible training collapse issue, as the single-view supervision signal easily forms the self-consistent illusion, yielding the reward hacking.
18
+
19
+ **Co-rewarding** is a novel self-supervised RL framework that improves training stability by seeking complementary supervision from another views. Specifically, Co-rewarding is instantiated in two ways:
20
+ 1. **Co-rewarding-I**: A data-side instantiation that derives reward signals from contrastive agreement across semantically analogous questions.
21
+ 2. **Co-rewarding-II**: A model-side instantiation that maintains a slowly-updated reference teacher with pseudo labels to realize self-distillation.
22
+
23
+ Intuitively, such instantiations introduce different levels of discrepancy to increase the difficulty of training collapse on trivial reasoning solutions. Empirically, Co-rewarding exhibits stable training across various setups, and outperforms other self-rewarding baselines by $+3.31\%$ improvements on average on multiple mathematical reasoning benchmarks, especially by $+7.49\%$ on Llama-3.2-3B-Instruct. Notably, Co-rewarding reaches or even surpasses RLVR with ground-truth (GT) label in several cases, such as a Pass@$1$ of $94.01\%$ on GSM8K with Qwen3-8B-Base.
24
+
25
+ For more details about the Co-rewarding method, including code and training scripts, please refer to the official [Github Repository](https://github.com/tmlr-group/Co-rewarding).
26
 
27
  ## Citation
28
+
29
+ If you use our datasets or models, please cite our paper:
30
+ ```bibtex
31
+ @article{zhang2025co,
32
+ title={Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models},
33
+ author={Zhang, Zizhuo and Zhu, Jianing and Ge, Xinmu and Zhao, Zihua and Zhou, Zhanke and Li, Xuan and Feng, Xiao and Yao, Jiangchao and Han, Bo},
34
+ journal={arXiv preprint arXiv:2508.00410},
35
+ year={2025}
36
  }
37
  ```