--- title: FineEnvs emoji: ๐Ÿค— colorFrom: yellow colorTo: purple sdk: static pinned: false license: mit --- ![FineEnv_banner](https://cdn-uploads.huggingface.co/production/uploads/6442d975ad54813badc1ddf7/pxJn0V8qoLyZAW6VFTt6R.png) [![๐Ÿ’ป Code](https://img.shields.io/badge/Code-FineEnvs-181717?style=for-the-badge&logo=github&logoColor=white)](https://github.com/adithya-s-k/FineEnvs) [![๐Ÿ“– Guide](https://img.shields.io/badge/Guide-The_Ultimate_Guide_to_RL_Environments-FFD21E?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/spaces/AdithyaSK/rl-environments-guide) [![๐ŸŽฅ Slides](https://img.shields.io/badge/Slides-RL_Environments_101-6B4FBB?style=for-the-badge&logo=huggingface&logoColor=white)](https://huggingface.co/spaces/AdithyaSK/rl-environments-101-slides) # ๐Ÿค— FineEnvs: Open RL Environments FineEnvs is a home for **end-to-end RL environment recipes**, built to make it easier to **explore, reproduce, train, and evaluate agent systems**. Explore complete and reproducible environment projects from us and the community, including: * ๐ŸŒ **Open RL environments** * ๐Ÿงฉ **End-to-end environment recipes** * ๐Ÿ’ป **Complete implementations** * ๐Ÿ“ฆ **Models, datasets, and artifacts** * ๐Ÿงช **Training and evaluation setups** * ๐Ÿš€ **Demos and Spaces** * ๐Ÿ“š **Tutorials and guides** All the reproducible code โ€” environments, rollouts, training configs, notebooks, article and slide sources โ€” lives in one repo: **[github.com/adithya-s-k/FineEnvs](https://github.com/adithya-s-k/FineEnvs)**. The artifacts those produce live here on the Hub. # FineEnvs Projects A growing collection of open projects, environments, resources, and artifacts. | Project | What it is | Explore | | :------------------------- | :----------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------ | | **FineEnvs Academy** | Articles, guides, tutorials, slides, and hands-on resources for learning how to build RL environments and agent systems. | [Explore โ†’](https://huggingface.co/collections/FineEnvs/fineenvs-academy-6a942aed464a2eaa85047596) | | **Data Agent** | Training SLMs for data science with multi-harness RL environments. | [Explore โ†’](https://huggingface.co/collections/FineEnvs/data-agent-6a942bca75a18973f8285023) | | **MiMo-V2.6-RL in Harbor** | All 7,780 of Xiaomi's MiMo-V2.6 RL environments as Harbor tasks, plus an explorer to browse them and run graded rollouts. | [Explore โ†’](https://huggingface.co/collections/FineEnvs/mimo-v26-rl-in-harbor-6ab836dba014bd7580f8a3fe) ยท [Explorer โ†’](https://huggingface.co/spaces/FineEnvs/MiMo-RL-Envs-Explorer) | | **Repo2RLEnv** | Verifiable coding and terminal RL environments in Harbor format, with per-task quality labels and provenance. | [Explore โ†’](https://huggingface.co/collections/FineEnvs/repo2rlenv-verifiable-rl-environments-6aa82300d7494c050f50508d) | Each numbered project below is a self-contained recipe: an environment, a training run, and every artifact it produced. | # | Project | What it is | Explore | | :-- | :--- | :--- | :--- | | **00** | **RL Environments 101** | Three environments implemented six times over, one per framework. Same logic, six dialects. | [Source โ†’](https://github.com/adithya-s-k/FineEnvs/tree/main/00-environments-101) ยท [Collection โ†’](https://huggingface.co/collections/FineEnvs/rl-envs-101-6abca516b22e765e0b1aa0d1) | | **01** | **LaTeX OCR** | Qwen3-VL-2B trained to read rendered math into LaTeX, scored by a reward served from a live Space. | [Collection โ†’](https://huggingface.co/collections/FineEnvs/latex-ocr-6aa7ed8498b3ffd222ad1c8c) | | **02** | **Watercolour** | Qwen3.5-35B-A3B trained to paint watercolours by writing p5.brush sketches, rewarded by taste rather than correctness. | [Collection โ†’](https://huggingface.co/collections/FineEnvs/paint-with-code-6a955b79d63f67f1631d9be6) | | **03** | **GeoGuesser** | A multi-turn visual geolocation environment, and the 4B trained on it until it outscored `gpt-5.4-mini` and `claude-haiku-4.5`. | [Collection โ†’](https://huggingface.co/collections/FineEnvs/geoguesser-env-6a969f8db267fe0e85fa1ab6) | | **04** | **SmolDataEnvs** | 5.5K+ data-analysis tasks for hill-climbing small models, graded deterministically with no LLM judge: plain prompts, verified SFT traces, and Harbor task suites. | [Collection โ†’](https://huggingface.co/collections/FineEnvs/smoldataenvs-6ab4f2f6e09b7cb872ebc867) | | **05** | **SmolDataEnvs: Multi-harness RL** | One small model trained with GRPO inside four unmodified coding agents (OpenCode, Claude Code, Codex, Mini-SWE-Agent), with LFM2.5-2.6B and Qwen3.5-2B checkpoints. | [Article โ†’](https://huggingface.co/spaces/FineEnvs/multi-harness-rl) ยท [Slides โ†’](https://huggingface.co/spaces/FineEnvs/multi-harness-rl-slides) ยท [Collection โ†’](https://huggingface.co/collections/FineEnvs/smoldataenvs-multi-harness-rl-6abdfaaa8d74dacd481d5212) | | **06** | **Multilingual** | Two OpenEnv servers: a million document pages in 22 languages with Sarvam Indic OCR Bench, and read speech in all 102 FLEURS languages. Gemma 4 trained on each to read and to hear Kannada. | [Collection โ†’](https://huggingface.co/collections/FineEnvs/multilingual-multimodal-envs-6ac0c27c137f93e0799603e4) | | **07** | **PortSimEnv** | Re-plan a broken week of container-ship dockings at the Port of Barcelona, built from the port's real 2024 records and graded against a plan CP-SAT proves optimal. | [Article โ†’](https://huggingface.co/spaces/FineEnvs/simulation-rl-environments) ยท [Play โ†’](https://huggingface.co/spaces/FineEnvs/PortSimEnv) ยท [Collection โ†’](https://huggingface.co/collections/FineEnvs/simulation-rl-envs) | # Articles & Talks | | What it covers | Read / Watch | | :--- | :--- | :--- | | ๐Ÿ“– **The Ultimate Guide to RL Environments** | Building and scaling RL environments in the LLM era โ€” how frameworks are built, how rewards are wired, how they scale to thousands of concurrent sessions. | [Read โ†’](https://huggingface.co/spaces/AdithyaSK/rl-environments-guide) | | ๐ŸŽž๏ธ **RL Environments 101** | From "what is an env?" to training your own: RL fundamentals โ†’ environment anatomy โ†’ OpenEnv โ†’ training with TRL. | [Watch โ†’](https://huggingface.co/spaces/AdithyaSK/rl-environments-101-slides) | | ๐Ÿ“ˆ **Scaling RL for LLMs** | RL environments and RL training โ€” what an environment is, how reward hacking happens, how to train against your own. AMD AI Dev Day. | [Watch โ†’](https://huggingface.co/spaces/AdithyaSK/scaling-rl-for-llms-amd-ai-dev-day) | | ๐Ÿ”€ **Multi-Harness Training** | OpenEnv ร— Harbor โ€” why an environment's failure model decides whether it can be trained against. | [Watch โ†’](https://huggingface.co/spaces/AdithyaSK/multi-harness-training-slides) | | ๐Ÿงญ **The Ultimate Guide to Multi-Harness RL** | Training small models on SmolDataEnvs with the same tasks and reward but a different tool loop each time (TRL, native OpenCode, Harbor), and what changes. | [Read โ†’](https://huggingface.co/spaces/FineEnvs/multi-harness-rl) | | ๐ŸŽค **Training a Coding Agent Through a Harness You Did Not Write** | Multi-harness RL talk by Sergio Paniego Blanco: one model, four unmodified coding agents, GRPO. | [Watch โ†’](https://huggingface.co/spaces/FineEnvs/multi-harness-rl-slides) | | ๐ŸŒ **How to turn a game into an RL environment** | The technical intuition, end to end: curating the data, designing the environment, shipping it with OpenEnv, and training a 4B against it with TRL. | [Read โ†’](https://huggingface.co/spaces/FineEnvs/geoguesser-article) | | ๐Ÿšข **Simulation RL Environments** | Turning real-world work into RL environments. Part 1 builds PortSimEnv from the Port of Barcelona's 2024 records and grades every plan against a proven optimum. | [Read โ†’](https://huggingface.co/spaces/FineEnvs/simulation-rl-environments) | # Environments Three reference environments, each implemented across six frameworks โ€” `openenv`, `ors`, `nemo_gym`, `verifiers`, `skyrl_gym`, `gem`. Same logic, six dialects. [Source โ†’](https://github.com/adithya-s-k/FineEnvs/tree/main/00-environments-101) ยท [RL Envs 101 collection โ†’](https://huggingface.co/collections/FineEnvs/rl-envs-101-6abca516b22e765e0b1aa0d1) | Environment | Tools | OpenEnv | ORS | NeMo Gym | | :--- | :--: | :--- | :--- | :--- | | **Jupyter agent** โ€” real code execution in an E2B sandbox | 4 | [Space](https://huggingface.co/spaces/AdithyaSK/jupyter-agent-openenv) | [Space](https://huggingface.co/spaces/AdithyaSK/jupyter-agent-ors) | [Space](https://huggingface.co/spaces/AdithyaSK/jupyter-agent-nemo-gym) | | **Wordle** โ€” multi-turn, pure Python, no backend | 1 | [Space](https://huggingface.co/spaces/FineEnvs/wordle-openenv) | [Space](https://huggingface.co/spaces/FineEnvs/wordle-ors) | [Space](https://huggingface.co/spaces/FineEnvs/wordle-nemo-gym) | | **Desktop** โ€” computer-use, vision-driven Linux desktop | 19 | [Space](https://huggingface.co/spaces/AdithyaSK/desktop-openenv) | [Space](https://huggingface.co/spaces/AdithyaSK/desktop-ors) | โ€” | # Build your own Five agent skills turn a plain-English description into a runnable RL environment across four frameworks โ€” works with Claude Code, Cursor, Codex, OpenCode, Gemini CLI and others. ```bash npx skills add adithya-s-k/FineEnvs ``` **We're looking for new end-to-end recipes** โ€” a task, an environment, a training run, and honest results. [Contributing guide โ†’](https://github.com/adithya-s-k/FineEnvs/blob/main/CONTRIBUTING.md) ## Citation ```bibtex @misc{fineenvs, author = {Kolavi, Adithya S}, title = {FineEnvs: Open Source RL Environments for LLM Agents}, year = {2026}, url = {https://github.com/adithya-s-k/FineEnvs} } ```