FineEnvs/geoguesser-qwen3.5-4b-grpo

GeoGuesser · Qwen3.5-4B · GRPO run 1

A LoRA adapter that plays GeoGuessr: dropped at a random street corner on Earth, it works out where it is and commits to a coordinate. Trained with GRPO against a live OpenEnv environment serving real Mapillary panoramas.

This is run 1, step 1000, and it is the adapter behind every number in the write-up.

What it scores

200 held-out tasks across 73 countries, pass@4, 800 episodes per arm, all scored through the same curve. The full board is in the project README.

model	mean-of-4	best-of-4	median error
claude-sonnet-5	0.6952	0.8397	324 km
this adapter (Qwen3.5-4B + LoRA)	0.6445	0.7092	662 km
gpt-5.4-mini	0.5732	0.7719	753 km
claude-haiku-4.5	0.5374	0.7014	939 km
Qwen3.5-122B-A10B	0.5338	0.6987	767 km
Qwen3.5-4B, untrained	0.4825	0.6589	1226 km

Against its own base, paired per task: +0.1620 ± 0.0137 (95% CI), better on 169 of 200 tasks. Country identified 21.2% → 33.1%. Episodes floored at zero on the environment's own curve 29.5% → 0.5%.

Scores are on the training reward curve, min(1, 0.5·e^(-d/1492.7) + 0.5·e^(-d/5000)) × (1 - min(cost, 0.2)), recomputed from raw distance. That is not GeoGuessr's own curve, which floors at zero past roughly 3,500 km.

How it behaves, which is the interesting part

It stopped exploring. Turns per episode fell from 6.7 to 1.1, output tokens from 1,062 to 66, and accuracy went up. It recognises the region from the first frame and commits to a plausible city in it, rather than reading signs across a dozen turns. That is not a random guess (a random guess on land averages roughly 8,000 km of error; this is 662 km) and it is not reward hacking: no identity leak in 882 audited prompts, no repeated tasks, and an action cost of 0.002 on the converged policy. It is what the reward rewarded, and the write-up is about why.

Training

base	Qwen/Qwen3.5-4B
method	GRPO (TRL), environment_factory multi-turn tool calling
LoRA	r=16, α=32, dropout 0.05, on q/k/v/o_proj
steps	1000, one task per optimizer step, NUM_GENERATIONS=8
rollouts	8,000 episodes over 1,000 distinct tasks
turns	12 max
image	448 px
optimiser	LR 3e-5, temperature 1.0, beta=0, scale_rewards="group"
hardware	4×A100 80 GB (one A100 also works), 10.2 hours, about $100
environment	HuggingEnvs/geoguesser-env, over HTTP

Note that it plateaued by step 200 (0.6393 against step 1000's 0.6445, inside the noise). If you reproduce this, stop at 250.

Using it

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-4B", device_map="auto")
model = PeftModel.from_pretrained(model, "HuggingEnvs/geoguesser-qwen3.5-4b-grpo")
processor = AutoProcessor.from_pretrained("Qwen/Qwen3.5-4B")

Or serve it with vLLM and point the eval harness at it:

vllm serve Qwen/Qwen3.5-4B --enable-lora \
  --lora-modules run1=HuggingEnvs/geoguesser-qwen3.5-4b-grpo

Everything else

• The write-up, which explains every decision and what it cost to learn it
• The environment, playable in a browser
• The task splits and the imagery
• All four training runs in one dashboard
• Code and exact commands

Imagery is Mapillary, CC BY-SA 4.0.

Citation

@misc{fineenvs,
  author = {Kolavi, Adithya S},
  title  = {FineEnvs: Open Source RL Environments for LLM Agents},
  year   = {2026},
  url    = {https://github.com/adithya-s-k/FineEnvs}
}