Demystifying Reinforcement Learning in Agentic Reasoning

## 🎯 About This Repository This repository contains the **Qwen3-4B-RA-SFT** model weights, a 4B-sized agentic reasoning model that is finetuned with our 3k Agentic SFT dataset, based on [Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507). ## 🌟 Introduction In our work, we systematically investigate three dimensions of agentic RL: **data, algorithms, and reasoning modes**. Our findings reveal - 🎯 **Data Quality Matters**: Real end-to-end trajectories and high-diversity datasets significantly outperform synthetic alternatives - ⚡ **Training Efficiency**: Exploration-friendly techniques like reward clipping and entropy maintenance boost training efficiency - 🧠 **Reasoning Strategy**: Deliberative reasoning with selective tool calls surpasses frequent invocation or verbose self-reasoning We contribute high-quality SFT and RL datasets, demonstrating that **simple recipes enable even 4B models to outperform 32B models** on the most challenging reasoning benchmarks. ## 📦 Resources | **Type** | **Name** | **Link** | | --------- | ------------------- | ------------------------------------------------------------ | | 📊 Dataset | 3K Agentic SFT Data | [🤗 HuggingFace](https://huggingface.co/datasets/Gen-Verse/Open-AgentRL-SFT-3K) | | 📊 Dataset | 30K Agentic RL Data | [🤗 HuggingFace](https://huggingface.co/datasets/Gen-Verse/Open-AgentRL-30K) | | 🤖 Model | Qwen2.5-7B-RA-SFT | [🤗 HuggingFace](https://huggingface.co/Gen-Verse/Qwen2.5-7B-RA-SFT) | | 🤖 Model | **Qwen3-4B-RA-SFT** | [🤗 HuggingFace](https://huggingface.co/Gen-Verse/Qwen3-4B-RA-SFT) | | 🤖 Model | DemyAgent-4B | [🤗 HuggingFace](https://huggingface.co/Gen-Verse/DemyAgent-4B) | ## 📝 Citation ```bibtex @article{yu2025demystify, title={Demystifying Reinforcement Learning in Agentic Reasoning}, author={Yu, Zhaochen and Yang, Ling and Zou, Jiaru and Yan, Shuicheng and Wang, Mengdi}, journal={arXiv preprint arXiv:2510.11701}, year={2025} } ```