---
license: apache-2.0
datasets:
  - openbmb/viper-train
language:
  - en
---

# Model Card for RLPR

[GitHub ](https://github.com) | [Paper](https://arxiv.org)

**RLPR-7B** is trained based on Qwen 2.5 7B with the novel [RLPR](https://github.com) framework.
\[placeholder]


## Model Details

### Key Features

*   💡 **Verifier-Free Reasoning Enhancement:** RLPR pioneers reinforcement learning for reasoning tasks by leveraging the LLM's intrinsic generation probability as a direct reward signal. This eliminates the need for external verifiers and specialized fine-tuning, offering broad applicability and effectively handling complex, diverse answers.
*   🛠️ **Innovative Reward & Training Framework:**
    *   Features a robust **Probability-based Reward (PR)** using average decoding probabilities of reference answers for higher quality, debiased reward signals, outperforming naive sequence likelihood.
    *   Implements an **adaptive curriculum learning** mechanism that dynamically filters prompts to stabilize training and significantly boost final performance.
*   🚀 **Leading Performance in General & Mathematical Reasoning:** Demonstrates substantial reasoning improvements across diverse benchmarks (e.g., 56.0 on MMLU-Pro, 55.4 on TheoremQA with Qwen2.5-7B). RLPR surpasses strong models reliant on external verifiers (like General Reasoner-7B) and other verifier-free approaches (like VeriFree).



![image/png](https://cdn-uploads.huggingface.co/production/uploads/65228f98aab6fd5585722875/juvqWbLV1HeNOq5BkKu-6.png)

### Highlights

![image/png](https://cdn-uploads.huggingface.co/production/uploads/65228f98aab6fd5585722875/lAWYIvgF7Qln7ql6J1JV7.png)

![[Pasted image 20250617081646.png]]
Existing RLVR methods rely on specialized verifiers for each domain, suffering from high complexity and limited scalability. Our RLPR framework, which replaces the complex verifier-based reward with a simple probability-based reward generated by the policy model $π_θ$. $Q$: input question, $z$: generated reasoning content before final answer, $y$: generated final answer, $y^∗$: reference answer. As shown in the example on the right side, rules and verifier models wrongly label both $y_2$ and $y_3$ as incorrect due to their limited capability of handling natural language complexity.

### Model Description
- **Trained from model:** [Qwen2.5-7B](https://huggingface.co/Qwen/Qwen2.5-7B)
- **Trained on data:** [VIPER-Train](https://huggingface.co/datasets)

## Usage
Please look at [GitHub](https://github.com) for more details about usage.


## Citation

If you find our model/code/paper helpful, please consider cite our papers 📝:

```bibtex
@article{placeholder,
  title={SCALING RLVR TO GENERAL DOMAIN WITHOUT VERIFIERS}, 
  author={placeholder},
  journal={placeholder},
  year={2025},
}
```