--- license: apache-2.0 base_model: Qwen/Qwen3.5-0.8B-Base base_model_relation: finetune library_name: transformers language: - en - pl - de - fr - es - it - pt - nl - cs - uk tags: - decision-making - routing - mixture-of-experts - moqle - lora - agents - e2e-testing - games - multilingual - on-device - system-one ---

▶ Live demo (previous version) · qev-0.8b

# squad-0.8b **A team of models. On your device. In milliseconds.** *squad: the qev model and its team of experts. Not related to the SQuAD question-answering dataset.* A 0.8B typed-decision model (System One API: a state plus typed questions in, a probability per allowed answer out, one forward pass, no text generated) built as a **MoQLE**, a *Mixture of Quantized LoRA Experts*: one trunk, a 1.1 MB router after layer 5, and 2-bit LoRA experts it calls in when a question needs them. **This revision (October 2026)** has a new trunk and four experts: **games**, **e2e** (decisions of a test agent driving a web app), **multilingual** (10 languages) and **documents**. - **Team:** each expert is 2.7 MB; the four add about 11 MB to the trunk. - **Fast:** 3.7 ms median per decision on one L40S (single requests), about 110 decisions/s with 16 clients. The router and experts cost nothing measurable. - **Better** than the previous squad-0.8b on rule following (+10 points on Rules-7), consistency (+5.8 on JevBench coherence), game boards (+3.9 on JevSpatial) and real web-app tests (3.4x more passed); slightly lower on Benchmark Heaven (−0.5) and games_bench (−0.9), both within run-to-run noise. ## Results All numbers measured with the same benchmark client; each model served by its own runtime (this repo's `qev_fast` engine, per-question requests). basal-omni-pico (Remek/basal-omni-pico) was served by basal 2.4.0, fast mode. | | squad-0.8b (previous) | **squad-0.8b (this revision)** | basal-omni-pico | |---|---|---|---| | Benchmark Heaven (public, 231) | 77.1 | 76.6 | 75.8 | | JevExtra | 71.3 | 71.7 | 64.6 | | JevSpatial | 75.8 | **79.7** | 68.4 | | Rules-7 | 63.8 | **74.1** | 69.5 | | JevBench accuracy | 84.9 | **86.4** | 84.0 | | JevBench coherence | 70.6 | **76.4** | 74.2 | | KLEJ (Polish, 7 tasks, 500 test items each) | 70.9 (trunk only) | 73.4 | **75.3** | | games_bench (9 unseen games, closed loop; 0 = random, 100 = planner) | 11.3 | 10.4 | 7.5 | | e2e on 8 real web apps (61 tests × 3 runs) | 10 / 183 | **34 / 183** | – | Speed on one NVIDIA L40S (Benchmark Heaven requests, one question each, after warm-up): | | this revision | basal-omni-pico (fast) | basal-omni-pico (eager) | |---|---|---|---| | single request, median / p95 | **3.7** / 30.5 ms | 6.0 / 31.5 ms | 50.4 / 101.3 ms | | 16 requests at once | 115 decisions/s | 94 decisions/s | 33 decisions/s | | GPU memory | 6.3 GB | – | – | **Read these honestly:** - Benchmark Heaven moves by about ±1 point between runs (one item is 0.43 points); its hard tier (111 items) is around 57% for every model here. - Rules-7: part of the gain comes from procedurally generated rule-application data whose layout (the policy inside the question) resembles Rules-7's. No benchmark text was used; labels are computed by code. Sub-scores vary across training seeds. - The e2e expert is used only when the router gives it at least 35% (a serving setting chosen while looking at Benchmark Heaven and the real-app tests). It still fails the TodoMVC apps (it clicks instead of pressing Enter). - games_bench is our own closed-loop benchmark of games never trained on; all models are close to random when a game is shown as a raw grid, coordinates or prose. - **Image input was not re-evaluated** for this revision (the trunk changed; the vision tower is unchanged). - Benchmark scores were used for reporting; checkpoints were chosen on our own validation data, except the serving threshold above. ## How it works - **Trunk:** a full fine-tune of Qwen3.5-0.8B-Base (24 layers, Gated DeltaNet + gated attention). It continues the previous qev-0.8b trunk on procedurally generated, exactly labelled data: consistency twins (the same decision asked in rephrased, reordered, merged or complementary forms, trained with a consistency loss), rule application with natural "up to / below" semantics, graded similarity, plus replay of the old data. Final weights are a weight average of three runs. No LLM-written text is in the trunk. - **Experts:** rank-64 LoRA on `gate_proj`, `up_proj`, `down_proj` of layers 12–23, `y = W x + 2.0 · B A x`, stored as 2-bit rows (`specialists/slots_2bit.safetensors`). Slot 1 games, 2 e2e, 3 multilingual, 4 documents. Trained by reinforcement learning only (closed-loop games, simulated web apps, decision tasks with a proper-scoring reward), trunk frozen. The multilingual and documents experts learned from data written and cross-checked by two LLMs. - **Router:** `LayerNorm → Linear(1024, 256) → GELU → Linear(256, 9)` on the mean RMS-normalised residual after layer 5 (`specialists/router.safetensors`), top-1 of {trunk, games, e2e, multilingual, documents}. Trained on measured outcomes (log-probability of the right answer) on validation rows no expert saw, so it only calls an expert where it helps. `router.min_p_slots` in the manifest holds per-expert confidence thresholds (e2e: 0.35). ## Quickstart (NVIDIA GPU) ```sh pip install -r requirements.txt -r requirements-cuda.txt ``` ```python from qev import QEV m = QEV.from_pretrained("labqoat/squad-0.8b", specialists=True) # router + experts (fast CUDA engine) m.decide(state, questions, per_question=True) # System One answers m.last_routes # e.g. ["games"] ``` ```sh python qev_server.py --model labqoat/squad-0.8b --specialists # POST http://127.0.0.1:8017/v1/systemone ``` Without `specialists=True` you get the trunk alone (not the old qev-0.8b: the trunk changed in this revision; pin the previous revision `2043fda` for it). Experts need the fast CUDA engine (`requirements-cuda.txt`). **The MLX and GGUF repos still hold the previous version** and will be updated separately. ## Files `model.safetensors` (new trunk; embeddings, head and vision tower as before) and tokenizer / config files · `specialists/` (manifest, router, 2-bit experts) · `qev.py`, `qev_server.py`, `qev_fast/` (runner, server, CUDA engine with experts) · `qev_image.py`, `qev_image_server.py` (transformers image path, trunk only) · `SHA256SUMS`. ## Limitations Text decisions only were evaluated for this revision. The e2e expert is far from reliable on real apps (34 of 183 real-app test runs pass). Polish sentiment and QA (KLEJ) are behind basal-omni-pico. Probabilities are calibrated with the previous version's temperatures; check them on your own data before using them as error rates. License: Apache-2.0.