---
title: Geoguesser Environment
emoji: 🌍
colorFrom: green
colorTo: blue
sdk: docker
pinned: false
app_port: 8000
base_path: /web
tags:
  - openenv
  - reinforcement-learning
  - geolocation
  - mcp-server
---

# GeoGuesser

A GeoGuessr-style visual geolocation environment. The agent is dropped at an
unknown street-level location, looks around, walks along the road, pins
candidate coordinates on a map to check itself, and commits to a final guess.
Reward is distance-based, using the game's own scoring curve.

Independent open-source project, unaffiliated with GeoGuessr AB. Imagery comes
from Mapillary contributors under CC-BY-SA-4.0.

## Quick start

```bash
cd envs/geoguesser_env

# The frozen 200-task eval split is committed, so this runs as-is, with the
# same configuration the Space uses.
./scripts/serve_local.sh             # http://localhost:8000/web/

# To build your own data (needs a free Mapillary token with READ scope):
export MAPILLARY_API_KEY_TRAIN="MLY|..."
python scripts/harvest_tiles.py                        # enumerate sequences
./scripts/build_dataset.sh                             # mirror tasks offline
python scripts/verify_offline.py tasks/pool_offline_5k.jsonl
python scripts/split_tasks.py tasks/pool_offline_5k.jsonl --eval 200
```

```python
from geoguesser_env import GeoGuesserEnv, GuessAction, LookAction, PinAction

env = GeoGuesserEnv(base_url="http://localhost:8000")

result = env.reset(split="eval", index=7)     # byte-identical on repeat
print(result.observation.prompt)

result = env.step(LookAction(heading_deg=90, fov_deg=45))
result = env.step(PinAction(lat=-16.5, lon=-68.1))
print(result.observation.feedback)
# Pin 1 placed at -16.5000, -68.1000 - Bolivia (South America).
# Nearest major city: La Paz, ~5 km E. 10 actions left.

result = env.step(GuessAction(response="Altiplano. <guess>-16.49, -68.12</guess>"))
print(result.reward, result.observation.distance_km)
```

## Tools

| Tool | What it does | Cost |
|------|--------------|------|
| `look(heading_deg, pitch_deg, fov_deg)` | Render a view. Heading is absolute, `0` is true north | −0.01 |
| `pan(delta_deg)` | Turn relative to the current heading | −0.01 |
| `zoom(fov_deg)` | Narrow the field of view; around 30 reads distant signs | −0.01 |
| `move(direction, meters)` | Walk the captured road; reports distance actually travelled | −0.05 |
| `place_pin(lat, lon, label)` | Pin a candidate and see where it falls on the map | −0.02 |
| `view_map(lat, lon, span_deg)` | Pan and zoom the map without pinning | −0.01 |
| `list_pins()` / `clear_pins()` | Review or drop candidates | free |
| `measure(lat_a, lon_a, lat_b, lon_b)` | Distance between two of your own points | free |
| `reverse_geocode(lat, lon)` | Name the country and nearest city at a coordinate | free |
| `submit_guess(lat, lon, ...)` | Commit the answer. Terminal | — |

Tools the backend cannot serve are **not registered**, so the agent never sees
a tool that always fails.

### The agent's map and the player's map agree

The player sees live OpenFreeMap tiles; the agent sees an offline Natural Earth
render. They have to agree about *how precisely a pin can be aimed*, because
that is what the distance reward measures — a map showing only country outlines
lets you place a country, not a point within a city.

So the guess map is zoom-aware. `place_pin` takes `span_deg`, and the render
adds detail as the window tightens:

| Window | What the agent's map shows |
|--------|----------------------------|
| wider than ~4 deg | coastlines, borders, country names |
| under ~4 deg | urban areas, highways, rivers, town names (Natural Earth 10m) |
| under ~0.35 deg | **real OSM streets**, fetched from Overpass and cached |

Natural Earth tops out at highway level — it shows the motorways around a city
but not the grid inside it. Below 0.35 degrees the map therefore fetches actual
ways from Overpass, generalising by zoom the way a real style does: minor
classes appear only once the window is tight enough to hold them, and widths
grow as it shrinks. A pin on Abuja at `span_deg=0.05` came back as an 11 km
window with the full street grid, drawn white-on-pale to read like the
player's Positron tiles.

Overpass has real limits, and they are the binding constraint on how this
scales: roughly **10,000 requests and 1 GB per day**, about **2 concurrent
slots per IP**, a 180 s runtime and 512 MiB memory ceiling per query, HTTP 429
when rate limited and 504 when a query is too large. Cooldowns lengthen for
heavy users. So street detail is right for eval, demos and modest training, and
the cache is what keeps it polite — a run doing millions of pins must pre-warm
or bundle a Protomaps extract instead.

The first render of a neighbourhood costs 3-16 s; every later one is served
from `data/geo/osm_cache/` in ~30 ms and is byte-identical. That makes an
episode deterministic once warm, and a frozen eval should pre-warm the cache
the same way it pre-warms panoramas — or set
`GEOGUESSER_STREET_DETAIL=0`, which falls back to Natural Earth and never
touches the network. Any fetch failure degrades to no streets rather than
failing the step.

The optional detail layers are fetched once, since 87 MB of GeoJSON does not
belong in the repo:

```bash
python scripts/fetch_detail_geo.py     # compacts to ~39 MB, gitignored
```

Without them the map still renders, with outlines and major cities only.

In the play page the pin carries the zoom you are actually looking at, so the
"what the agent sees" panel is framed like your own view at the same scale and
with comparable detail.

Overpass has a usage policy that discourages heavy automated querying, so this
is right for eval, demos and modest training, and the cache is what keeps it
polite. A run doing millions of pins should pre-warm or bundle a Protomaps
extract instead.

### Zoom resolves real detail

Zooming is not cosmetic, but it needs the right source. A 30-degree view of a
2048x1024 panorama samples only about 170 source pixels, so narrowing the field
of view barely adds information — measured mean gradient 6.60 at 90 degrees
against 7.03 at 30. The 7680x3840 original roughly doubles it (10.23 against
14.87), which is the difference between guessing at a sign and reading it.

So each panorama is cached twice. Wide views render from the 2048 derivative in
~30 ms; a field of view at or below 45 degrees pulls the original and renders in
~70 ms. If no original exists the step degrades to a soft view rather than
failing. Set `GEOGUESSER_HIRES_ZOOM=0` to disable it.

### Pinning tells you where you pointed, not whether you are right

`place_pin` returns a rendered map and a description of the pinned location:
country, subregion, nearest city with distance and bearing, and the distance
to the agent's own earlier pins. It reveals nothing about the target.

That restraint is deliberate. Any signal about the truth — a distance, a
warmer/colder hint — would make binary search the optimal policy, and the
environment would measure bisection rather than geographic reasoning. Distance
and score arrive only from `submit_guess`.

## Reward

```
geo     = exp(-distance_km / 1492.7)           # GeoGuessr's curve, in [0, 1]
partial = 0.15 * country_hit + 0.10 * region_hit   # when hierarchical
cost    = 0.01*looks + 0.01*maps + 0.02*pins + 0.05*moves
reward  = clip(geo + partial, 0, 1) - cost
```

An unparseable or out-of-range guess scores `0.0` and says why. Parsing
accepts what models actually emit: decimal pairs, DMS (`48°51'29"N`), labelled
`lat:`/`lon:`, JSON, and `<guess>` tags.

### Training against it

The defaults above reproduce the game, which makes a score directly comparable
to GeoGuessr. They are the wrong shape for RL, and three flags change that:

| flag | play / eval | training | why |
|---|---|---|---|
| `reward_shape` | `"geoguessr"` | `"mixture"` | The game curve is worth **0.018** across the whole 6000-20000 km range, so a policy gets no gradient for landing on the right continent instead of the wrong one. `"mixture"` adds a 5000 km scale, making that span worth 0.150. |
| `cost_mode` | `"subtract"` | `"multiply"` | Mean action cost for a 4B model is 0.13 and the curve falls below that at ~3300 km, so `max(0, geo - cost)` floors every worse guess at exactly zero. Measured over 200 episodes: **77 collapsed to 0.0 with zero variance**, so a GRPO group drawn from them has no advantage and yields no gradient. A multiplier cannot do this. |
| `hide_task_identity` | `False` | **`True`** | `metadata` carries `attribution.creator_username`, and the contributor determines the country outright for **74% of training tasks** (`amsterdam` only maps the Netherlands). `task_index`/`task_id`/`sequence_id` are a few thousand memorisable keys straight to a coordinate. Either lets a policy score without reading the image. |

```bash
GEOGUESSER_REWARD_SHAPE=mixture \
GEOGUESSER_COST_MODE=multiply \
GEOGUESSER_HIDE_IDENTITY=1 \
    uvicorn geoguesser_env.server.app:app
```

Replaying all 3,037 recorded eval episodes through both settings: episodes
scoring exactly zero fall from **10-46% to 0%** for every model, and the
leaderboard order only changes within the tiers already documented as inside
noise at n=200.

The terminal observation carries full provenance either way — once the truth is
revealed it can no longer be used to shortcut the episode — so recorded traces
stay complete under `hide_task_identity`.

## Baselines

Measured with `examples/geoguesser_llm_rollout.py` on the committed index,
tasks 0/7/14/21/28, so the numbers are reproducible rather than illustrative.
Five episodes is far too few for a leaderboard; they are a smoke test that the
task is solvable and the reward is discriminative.

| Model | Mode | Mean reward | Median distance | Within 200 km | Parsed |
|-------|------|------------:|----------------:|--------------:|-------:|
| `claude-sonnet-5` | single-shot | **0.896** | 98 km | 4/5 | 5/5 |
| `claude-sonnet-5` | agentic, tasks 0-5 | 0.539-0.653 | 574-660 km | 3/6 | 6/6 |
| `Qwen/Qwen3.5-9B` | agentic, tasks 0-5 | 0.355 | 1,136 km | 0/5 | 5/6 |
| `Qwen/Qwen3.5-9B:together` | single-shot | 0.277 | 1,139 km | 1/3 | 3/5 |
| `Qwen/Qwen3.5-9B:together` | agentic, 6 turns | 0.304 | 579 km | 0/1 | 1/2 |
| `Qwen/Qwen3.5-9B:together` | agentic, 8k tokens | 0.087 | 2,423 km | 0/1 | **4/4 turns** |
| `Qwen/Qwen3.5-9B:together` | single-shot, 8 eps, 4 parallel | 0.412 | 787 km | 1/6 | 6/8 |

Sonnet placed two guesses within 2 km. The agentic score sits slightly below
single-shot on the same tasks because looking around costs reward and the extra
views did not always pay for themselves — which is the trade-off the
environment is meant to expose, not a defect.

Qwen does follow the multi-turn protocol: across the agentic runs it produced
`look`, `move`, `zoom`, `pin` and `guess` actions and navigated up to 68 m down
a road. Two things had to be right first, and both are prompting or plumbing
rather than capability:

- **Token budget.** Reasoning models put their chain of thought in a separate
  `reasoning_content` field and can exhaust the budget before emitting any
  content, which looks exactly like a model that cannot see images. At 1,024
  tokens Qwen scored 0/5 with empty replies; at 3,500 it followed the protocol
  intermittently, failing turns whose reply came back as pure reasoning; at
  8,000 it parsed 4/4 turns. The example defaults to 3,000 and takes
  `--max-tokens`.
- **A deadline.** Left to itself Qwen explored until the turn budget ran out and
  scored zero. The agentic prompt now warns it explicitly when two turns remain,
  after which it committed.

What remains is accuracy, not plumbing: its guesses landed 452 km, 579 km and
2,423 km out against Sonnet's 98 km median. It is also 10x slower — 110-193 s
per agentic episode against Sonnet's 10-16 s.

```bash
python examples/geoguesser_llm_rollout.py --provider anthropic \
    --model claude-sonnet-5 --episodes 5
python examples/geoguesser_llm_rollout.py --provider hf \
    --model "Qwen/Qwen3.5-9B:together" --episodes 5 --max-tokens 4000
python examples/geoguesser_llm_rollout.py --provider anthropic \
    --mode agentic --episodes 3 --verbose
```

## Parallel rollouts

An episode is stateful, so concurrent rollouts each need their own environment
instance over the shared read-only index and cache. `--concurrency` does that:

```bash
python examples/geoguesser_llm_rollout.py --provider hf \
    --model "Qwen/Qwen3.5-9B:together" --episodes 8 --concurrency 4
```

Eight Qwen episodes took 113.6 s wall against 334.0 s of summed latency — a
**2.94x speedup** on 4 workers, the shortfall being the provider's own queuing
rather than the environment, which spends ~28 ms on a reset.

## Scaling: what the environment can actually supply

Measured on an 18-core machine with a warm cache and street detail off, one
environment per worker, with a correctness assertion in the loop so an
interference bug cannot masquerade as throughput.

Per-step cost is dominated by map rendering, not imagery:

| Step | Cost |
|------|-----:|
| reset, or `look` at 90 deg fov | **29 ms** |
| `look` at 30 deg fov, from the original | 74 ms |
| `place_pin`, a two-panel map | **207 ms** |
| `submit_guess` with the reveal map | 278 ms |

That shapes the two configurations:

| Config | Workers | Episodes/s | Notes |
|--------|--------:|-----------:|-------|
| training — views only, no pins, no reveal | 8 threads | **31.7** | 158 env steps/s |
| eval — pins and reveal map | 4 processes | **3.5** | matplotlib is GIL-bound |
| eval — pins and reveal map | 8 threads | 1.9 | threads do not help here |

Two facts fall out of that. `look` releases the GIL — the reprojection is numpy
and the encode is Pillow — so **threads scale well for view-only work** and
processes only add startup cost. Map rendering is pure Python, so it is
**GIL-bound and needs processes**, which buy about 1.8x before contention.

`GEOGUESSER_REVEAL_MAP=0` is the single biggest throughput lever: a
training-shaped episode drops from **389 ms to 117 ms**, since the reveal map is
280 ms that a training run never reads — the reward and the distance are in the
observation either way. Keep it on for evals, traces and the UI.

Memory is small: about 215 MB for the imports, 100 MB more once the geodata
caches fill, and **~13 MB per additional environment in the same process**. The
Natural Earth layers are process-shared through an LRU cache, so threads are far
cheaper than processes here too.

### Rate limits that actually bind

| API | When it is called | Limit |
|-----|-------------------|-------|
| Mapillary | only on a cache miss | 60,000/min entity, 10,000/min search, 50,000/day tiles |
| Overpass | only a pin below 0.35 deg span | ~10,000/day, 2 concurrent slots per IP |
| the model | every turn | the real constraint |

With a warm cache the environment makes **no network calls at all**. Default
pins use a 7 degree span, which is above the street threshold, so they do not
touch Overpass either — only a deliberately zoomed pin does. For an eval sweep
set `GEOGUESSER_STREET_DETAIL=0` unless you have pre-warmed, since 100 zoomed
pins against 2 concurrent slots would throttle immediately.

### 100-task eval

Sonnet agentic measured 13.1 s per episode, so 100 episodes cost about 1,310
model-seconds and the concurrency you can use is set by the provider, not by
this environment:

- concurrency 4 → **~5.5 minutes**
- concurrency 8 → **~2.7 minutes**, if your token-per-minute ceiling allows it

An agentic episode sends roughly five 640x640 views, about 550 tokens each, plus
a growing text prompt. At a 400k input-tokens-per-minute ceiling that is roughly
6 episodes in flight before tokens, not latency, become the limit — which is why
4 to 6 is the practical range for Sonnet. Qwen through the router reached 3.97x
on 6 workers, so 6 to 8 there.

Meanwhile the environment can supply 3.5 episodes/s in eval configuration
against the 0.3 to 0.6 episodes/s those concurrencies actually consume, so it
has roughly ten times the headroom it needs.

### 1000 training steps

For GRPO with 8 prompts and a group of 16, that is 128 episodes per step and
**128,000 episodes**, or about 640,000 env steps:

- environment time: 128,000 / 31.7 ≈ **1.1 hours total**, spread across workers
- imagery: nothing, with a warm cache
- storage: 100 tasks warm is 60 MB; 5,000 tasks would be about 1.5 GB of start
  frames

So the environment is not the constraint — 640,000 model calls are, which needs
batched local inference rather than an API. The constraint that *is* ours is
**task diversity**: 128,000 episodes over 100 tasks means each location is seen
1,280 times, which is memorisation territory. Before a run that long, either
harvest more tasks or add seeded heading augmentation, which multiplies
effective tasks 8 to 12 times from imagery already on disk.

## Readiness audit

`scripts/readiness_check.py` checks the properties that only appear at real
scale and concurrency, rather than on the four committed fixtures:

```bash
python scripts/readiness_check.py --full
```

```
[PASS] index integrity            100 tasks, 47 countries, 100 unique sequences
[PASS] all tasks render           100/100 rendered, median reset 28 ms
[PASS] cross-process determinism  two subprocesses and this process agree
[PASS] parallel isolation         8 concurrent episodes, each its own task
[PASS] offline with warm cache    12/12 served with fetching disabled
[PASS] reward is discriminative   uniform-random 0.029, fixed-point 0.111
[PASS] step latency               look 29 ms, pin + map 249 ms
[PASS] one guess per episode      a second guess returns reward=None
[PASS] pin never leaks the target 60 pins across 20 tasks revealed nothing
```

The reward check matters most: a uniform-random guesser scores **0.029** and the
best trivial constant guess **0.111**, against Sonnet's 0.896. The signal is
measuring geolocation rather than rewarding noise.

## Tracing a rollout

`--trace-dir` records every turn: the image the model saw, what it said, the
action it chose, the environment's reply, steps left and running cost. A second
script renders that as one self-contained HTML page, which is the difference
between knowing the reward and seeing why:

```bash
python examples/geoguesser_llm_rollout.py --provider anthropic \
    --mode agentic --episodes 6 --trace-dir rollouts
python scripts/render_trace.py rollouts/anthropic_agentic
```

Images are written beside the trace rather than inlined, since six agentic
episodes carry around 35 views and a JSONL with those in it is neither readable
nor loadable.

Every step carries an image, including the guess: a guess returns a **reveal
map** with the guess, the true location and the line between them. Truth is
drawn only there, after scoring. When the guess is more than 25 degrees out the
second panel frames the true location instead of both points, because squashing
a hemisphere into a panel shows nothing.

## On the Hub

| | |
|---|---|
| Space | [`HuggingEnvs/geoguesser-env`](https://huggingface.co/spaces/HuggingEnvs/geoguesser-env) |
| Space (mirror) | [`AdithyaSK/geoguesser-env`](https://huggingface.co/spaces/AdithyaSK/geoguesser-env) |
| Task splits | [`HuggingEnvs/geoguesser-tasks`](https://huggingface.co/datasets/HuggingEnvs/geoguesser-tasks) |
| Imagery | [`HuggingEnvs/geoguesser-panos`](https://huggingface.co/buckets/HuggingEnvs/geoguesser-panos) (Storage Bucket, public, mounted read-only at `/data`) |

The Space and the bucket deliberately live in different namespaces: moving the
Space should not mean re-uploading 22 GB, so `deploy_hub.py` takes the owner of
each separately (`GEOGUESSER_HF_ORG` and `GEOGUESSER_HF_BUCKET_OWNER`).

The same client drives either:

```python
env = GeoGuesserEnv(base_url="http://localhost:8000")                      # local
env = GeoGuesserEnv(base_url="https://huggingenvs-geoguesser-env.hf.space") # Space
```

Only the file locations differ — locally the indexes and imagery are in the
repo, on the Space they arrive through the bucket mount. Splits, step budget,
street labels and offline enforcement are identical, and verified so: the same
task returns the same image checksums, reward and distance from both.

Deploy with `scripts/deploy_hub.py --all`. `openenv push` is deliberately not
used: it cannot attach a bucket volume, and its default excludes would upload
22 GB of panoramas into git.

### Local and Space parity

Verified across a local server and both Spaces, on both splits, driven by the
same client. Every field below is identical in all three:

| | eval index 42 | train index 1234 |
|---|---|---|
| task id | `eval-00042` | `train-01234` |
| reset image sha256 | `a1447fd8` | `5494a500` |
| look image sha256 | `1c95af16` | `0850e712` |
| move distance | 27.778734090271577 m | 26.196924979479498 m |
| reward, distance | 0.0, 7463.03 km | 0.0, 12647.18 km |

The guess map is **pixel-identical** too: mean absolute difference 0.000/255
over 1020x390. Only the PNG encoding differs, not the content.

The one fragile part is the street layer, which comes from Overpass. Overpass
answers a laptop in ~2 s but intermittently returns **504 Gateway Timeout** to
datacenter egress, which once left the Space rendering coarse
Natural-Earth-only maps while a laptop drew the full labelled grid. A single
retry recovers it, and the environment now *reports* the state rather than
degrading silently: observation metadata carries `street_detail` as `on`, `off`
or **`unavailable`**, so a poorer map is visible instead of looking like a
styling choice. Scoring is unaffected either way -- reward is distance-based.
`GEOGUESSER_OSM_CACHE` points the street cache at a mounted, pre-warmed
directory for deployments that cannot reach Overpass at all.

## Task API

The environment implements the core `TaskProvider` protocol, so the Task API
routes core already registers become live. Task discovery is metadata only; it
never starts an episode.

```bash
curl localhost:8000/geoguesser_env/splits
# [{"name":"train","type":"train","num_tasks":3452,"default":true},
#  {"name":"eval","type":"test","num_tasks":200,"default":false}]

curl -X POST localhost:8000/geoguesser_env/num_tasks -d '{"split":"eval"}'
curl -X POST localhost:8000/geoguesser_env/task      -d '{"split":"eval","index":12}'
```

```python
env.list_splits()          # [{"name": "eval", "type": "test", ...}, ...]
env.num_tasks("eval")      # 200
env.get_task("eval", 12)   # metadata, no coordinates and no country
```

**Task specs are deliberately truth-free.** They carry `task_index`, `task_id`,
`split`, `n_frames`, `provider`, `sequence_id` and `offline_ready` — never
coordinates and never the country. A spec travels to whatever orchestrates a
run, and a label sitting in a spec can reach a prompt. The true location is
revealed in observation metadata after the guess, which is the one place it
belongs. Per-country eval breakdowns therefore come from finished episodes, not
from `list_tasks`.

Splits are invisible to the agent. `RESERVED_TOOL_NAMES` blocks a `reset` MCP
tool, so there is no way for a policy to see or choose its own task — the
"agents cannot reset" invariant.

## Collecting evals

`scripts/collect_eval.py` runs models against a split and records everything
about each episode. One JSONL line per episode, ~13 KB.

```bash
export ANTHROPIC_API_KEY=...
python scripts/collect_eval.py --provider anthropic --model claude-sonnet-5 \
    --split eval --max-turns 12

# several endpoints in one run, each at its own concurrency
cp models.example.json models.json     # anthropic / openai / HF router / vLLM
python scripts/collect_eval.py --models models.json --split eval
```

Output lands in `rollouts/<run_id>/`: `episodes.jsonl` plus a `run.json` with
per-model aggregates (mean reward, median distance, within-1/25/200/750 km,
parse rate, forced guesses, tokens, latency). Runs are **resumable** — re-run the
same `--run-id` and it skips episodes already recorded.

Each turn carries the camera state **before and after** the action, which is
what makes a rollout re-renderable: a pan from 0 to 270 degrees can only be
animated if both ends are known. It also carries the exact prompt, the raw
reply, separated reasoning, finish reason, token counts, per-attempt errors and
latency, and the provider's own response id.

**Pixels are not stored.** The environment is deterministic and the panoramas
are local, so a renderer replays the state trajectory instead. Storing them
would cost roughly 14 GB for 200 tasks across six models, all re-derivable. What
*is* stored is a sha256 per observation, so a replay can be *verified* rather
than assumed:

```bash
python scripts/verify_replay.py ../../rollouts/<run_id>/episodes.jsonl
# 3 episodes · 19 view turns checked · 19 match · 0 mismatch
# every view turn reproduces byte-for-byte
```

That check is not decorative: it exits non-zero on drift, because a video built
from a mismatched trace looks authoritative and shows something the model never
saw. Guess maps are excluded from it by construction — they depend on the
Overpass response of the moment. Use `--save-frames` when you want the literal
bytes anyway.

## Rollout video

Two steps, split where the work naturally divides. Python owns
pixels-from-panoramas, because the gnomonic reprojection already lives here and
is verified byte-exact against the trace. React owns layout, typography and
transitions, because that is where iterating on them is pleasant.

```bash
# 1. Frames + timeline.json, straight into the Remotion project's public/
python scripts/render_rollout.py ../../rollouts/<run>/episodes.jsonl --episode 0

# 2. Compose
cd video && npm install
npx remotion render Rollout out.mp4 --props=public/rollouts/<slug>/timeline.json
npx remotion studio          # iterate on the composition live
```

**The pan is a real pan.** A `look` from 0 to 270 degrees is not a cut between
two stills: it renders one intermediate gnomonic reprojection per video frame
along the *shortest angular path*, eased like a camera rather than swept
linearly — 350 to 10 degrees pans +20, not -340. Zoom interpolates the field of
view the same way. Verified: a `look` segment produces 29 distinct images, a
`zoom` 24, with no duplicates.

Layout is a large panorama viewport with a HUD of the state the agent is acting
on (heading, fov, actions left, cost), and a trace pane that reveals turns as
they happen — the active turn carries the model's raw reply, past turns recede.
Then a score card with the guess against the truth.

Fidelity is checked before anything is composed: every view keyframe is
re-rendered at the size the model saw and compared to the trace's sha256, and a
mismatch aborts. A video that looks authoritative while showing something the
model never saw is worse than no video.

## Reproducibility

```python
env.reset(split="eval", index=7)  # exact task, byte-identical    -> GRPO, eval
env.reset(split="train", seed=42) # tasks[42 % n_tasks]           -> replay
env.reset()                       # random task in the default split, split and
                                  # index both recorded in metadata -> UI
```

`task_index=` still works as an alias for `index=`, so trajectories recorded
before splits existed still replay. The split is recorded in observation
metadata: without it a bare index is ambiguous across three indexes, and a
trajectory stops being replayable.

Byte-identical repeats hold because panorama bytes come from a local cache
rather than an expiring CDN URL, reprojection is pure numpy with integer
sampling, and the initial heading is pinned to each panorama's own
`compass_angle`.

An eval score is only meaningful alongside its provenance — the env version,
the task index, and `GEODATA_VERSION` from `server/render/minimap.py`, since
the bundled vectors determine the reverse-geocode text the agent sees.

## Training and collection

The environment plugs into `openenv.core.harness`, so a rollout function and a
collector come for free:

```python
from geoguesser_env import GeoGuesserEnv
from geoguesser_env.harness import GeoGuesserSessionFactory, load_tasks

tasks = load_tasks("tasks/train_pano_v3.jsonl", repeat=16, split="train")
factory = GeoGuesserSessionFactory(
    lambda: GeoGuesserEnv(base_url="http://localhost:8000")
)
```

See `examples/geoguesser_rollout.py` for a scripted rollout and
`examples/geoguesser_collect.py` for JSONL collection with resume.

## Human play

A five-round game, 5,000 points a round on the same curve the environment
rewards, so a human score is directly comparable to GeoGuessr intuition and to
the agent's reward (both are shown).

**A round is one episode with one guess.** The five-round game is a UI wrapper
around five separate episodes; the environment itself never accepts more than
one guess, because `submit_guess` is terminal.

The page plays through the environment rather than simulating it. It opens the
same WebSocket session API a client uses, calls `reset(task_index=...)`, and
sends every pin, look, zoom and move as a real charged step — so the step
counter, the accumulated cost and the final reward are the environment's own
numbers, not the browser's. A side panel shows the observation stream an agent
would receive, including the environment's own rendered map and views.

Note that plain REST `/step` builds a fresh environment per request, so a
stateful episode has to run over `/ws`; the Python client does this already.

```bash
uv run --project . server
# then open http://localhost:8000/geoguesser/play
```

The page stands alone at `/geoguesser/play` and is also embedded in the Gradio
playground's **Custom** tab when the web interface is enabled:

```bash
ENABLE_WEB_INTERFACE=true uv run --project . server   # http://localhost:8000/web/
```

Pick a split and an episode with the `reset(split=)` and `reset(index=)`
controls above the game and press
**load episode**, or **random episode** — the same call an eval harness makes,
so you can replay exactly the episode an agent saw. Those controls live on the
Gradio side because choosing a task is orchestration, not something the player
does mid-round; the page itself reads `?task=` from its URL, so
`/geoguesser/play?task=42` opens that episode directly.

Drag to look around and scroll to zoom (free, for orientation). The `look()`
and `zoom(30)` buttons run charged environment steps and show what the agent
sees. Arrows, or the arrow keys, walk the road — the main view follows, keeping
your heading. **M** toggles a larger map, **T** the trace panel, **Enter**
submits and then advances. On submit the map takes the screen and draws the
line between guess and truth, exactly like the game; the result bar shows
distance, points, env reward and the true location, and a scoreboard breaks
down all five rounds at the end.

Panoramas are rendered by Pannellum and the map by MapLibre over OpenFreeMap
tiles — no API key, no request limits. The imagery credit line names the
Mapillary contributor, which the CC-BY-SA licence requires.

The page has to be a standalone document rather than a Gradio `gr.HTML`
fragment: `gr.HTML` inserts markup without executing `<script>` tags, so the
viewers never initialise and the panel renders blank with no error anywhere.

Extra routes, all local:

| Route | Returns |
|-------|---------|
| `/geoguesser/play` | the play page |
| `/geoguesser/tasks` | `{"n_tasks": N}` |
| `/geoguesser/task/{i}` | task metadata, including ground truth for the human UI |
| `/geoguesser/pano/{i}` | the starting equirectangular panorama |
| `/geoguesser/pano/{i}/{frame}` | one frame's panorama, so the viewer follows `move()` |

The human map uses live tiles; the agent's map stays the offline Natural Earth
render, so the agent keeps a determinism the browser does not need. Note that
`/geoguesser/task/{i}` exposes ground truth — it exists for a person playing in
their own browser, and agent observations still withhold it until the guess.

## Configuration

| Variable | Default | Meaning |
|----------|---------|---------|
| `GEOGUESSER_TASKS_EVAL` | `tasks/eval_pano_v3.jsonl` | Frozen eval split |
| `GEOGUESSER_TASKS_TRAIN` | `tasks/train_pano_v3.jsonl` | Training split |
| `GEOGUESSER_DEFAULT_SPLIT` | `train` | Split `reset()` uses when none is named |
| `GEOGUESSER_INDEX` | `tasks/pano_v1.jsonl` | Legacy single index, used only when no split resolves |
| `GEOGUESSER_CACHE` | `data/panos` | Panorama cache directory |
| `GEOGUESSER_EPISODE_MODE` | `agentic` | `agentic`, `single_shot` or `nmpz` |
| `GEOGUESSER_MAX_STEPS` | `24` | Actions before the episode is cut off |
| `GEOGUESSER_REWARD_MODE` | `coords` | `coords` or `country_only` |
| `GEOGUESSER_HIERARCHICAL` | `0` | Add country and region partial credit |
| `GEOGUESSER_VIEW_SIZE` | `640` | Edge length of rendered views |
| `GEOGUESSER_ALLOW_FETCH` | `1` | Whether a cache miss may reach the API |
| `GEOGUESSER_HIRES_ZOOM` | `1` | Render views at or below 45 deg fov from the original |
| `GEOGUESSER_STREET_DETAIL` | `1` | Fetch real OSM streets below 0.35 deg. Governs Overpass only, independent of `ALLOW_FETCH`, and caches to local disk |
| `GEOGUESSER_REVEAL_MAP` | `1` | Draw the guess-versus-truth map; `0` is 3x faster for training |
| `GEOGUESSER_OSM_CACHE` | `data/geo/osm_cache` | Street-window cache; point at a pre-warmed mount where Overpass is unreachable |
| `MAPILLARY_API_KEY` | — | Needed by the builder, and only on a cache miss |

## Data

Three splits, carved from one 3,673-task pool so contamination is enforced
exactly once, at split time, rather than reasoned about across two harvests:

| Split | Type | Tasks | Countries | Offline |
|---|---|---|---|---|
| `eval` | `test` | 200 | 73, capped at 4 each | all 24 frames mirrored |
| `train` | `train` | 3,452 | 132 | all 24 frames mirrored |
| `random` | `validation` | 1.2M pool rows | global | no, fetches on demand |

Separation follows the OSV-5M rule: no shared `sequence_id`, and no training
task within 1 km of an eval task. Frames sit ~3.3 m apart, so holding out an
image while keeping its neighbour holds out nothing. The split script verifies
its own work and exits non-zero if either rule is violated — the committed
split reports 0 shared sequences and a closest train task 1.07 km away.

| | |
|---|---|
| Frames per task | 23.2 mean (8 min, 24 max), ~3.3 m apart |
| Eval index | 1.2 MB, committed |
| Train index | 20 MB, in the Storage Bucket |
| Imagery | 22 GB for 86k frames, 0.26 MB mean per frame |

`eval` is committed because a frozen benchmark belongs in version control,
where a change to it shows up in review. The training index and the imagery
live in a Storage Bucket, mounted read-only at `/data` on a Space.

The `random` split is **not yet implemented** — the plumbing takes arbitrary
named splits, but the pool-backed sampler is still to come.

Each index is self-contained: every frame's coordinates, heading and capture
date live in the JSONL, so the movement graph resolves offline.
Only image bytes are fetched, and only on a cache miss, because Mapillary
`thumb_*_url` values are expiring signed URLs that cannot be stored.

Coverage is uneven and worth knowing about. Probing 45 Street-View
coordinates found any Mapillary imagery at 21 and a 360-degree panorama at
only 7, heavily clustered. Panorama-first discovery is therefore the only
approach that works — roughly 5% of probe points yield a usable sequence, so
reaching 100 tasks took two passes with different seeds, merged by
`scripts/merge_task_indexes.py`. Africa and Oceania are thin because 360-degree
contributors are; that is a property of the source, documented rather than
papered over.

## Known gaps versus the real game

Movement follows captured sequences and stops where one ends. There is no
multi-round cumulative score, no wall-clock timer (a step budget stands in for
it), and no satellite layer on the guess map. Coverage hints and web search are
deliberately excluded: the first is a crutch, the second turns the task into
retrieval.

See [DESIGN.md](DESIGN.md) for the reasoning behind these choices.