Instructions to use labqoat/squad-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use labqoat/squad-0.8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="labqoat/squad-0.8b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("labqoat/squad-0.8b")
model = AutoModelForMultimodalLM.from_pretrained("labqoat/squad-0.8b", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use labqoat/squad-0.8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "labqoat/squad-0.8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "labqoat/squad-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/labqoat/squad-0.8b

SGLang

How to use labqoat/squad-0.8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "labqoat/squad-0.8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "labqoat/squad-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "labqoat/squad-0.8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "labqoat/squad-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use labqoat/squad-0.8b with Docker Model Runner:
```
docker model run hf.co/labqoat/squad-0.8b
```

squad-0.8b / README.md

qmichalwarda

New trunk (soup of procedural-data continuations) + 4 experts (games, e2e, multilingual, documents) + router fitted on held-out rows; e2e confidence 0.35

58c22ba verified 11 days ago

preview code

Raw History Blame

7.08 kB

metadata

license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B-Base
base_model_relation: finetune
library_name: transformers
language:
  - en
  - pl
  - de
  - fr
  - es
  - it
  - pt
  - nl
  - cs
  - uk
tags:
  - decision-making
  - routing
  - mixture-of-experts
  - moqle
  - lora
  - agents
  - e2e-testing
  - games
  - multilingual
  - on-device
  - system-one

▶ Live demo (previous version) · qev-0.8b

squad-0.8b

A team of models. On your device. In milliseconds.

squad: the qev model and its team of experts. Not related to the SQuAD question-answering dataset.

A 0.8B typed-decision model (System One API: a state plus typed questions in, a probability per allowed answer out, one forward pass, no text generated) built as a MoQLE, a Mixture of Quantized LoRA Experts: one trunk, a 1.1 MB router after layer 5, and 2-bit LoRA experts it calls in when a question needs them.

This revision (October 2026) has a new trunk and four experts: games, e2e (decisions of a test agent driving a web app), multilingual (10 languages) and documents.

Team: each expert is 2.7 MB; the four add about 11 MB to the trunk.
Fast: 3.7 ms median per decision on one L40S (single requests), about 110 decisions/s with 16 clients. The router and experts cost nothing measurable.
Better than the previous squad-0.8b on rule following (+10 points on Rules-7), consistency (+5.8 on JevBench coherence), game boards (+3.9 on JevSpatial) and real web-app tests (3.4x more passed); slightly lower on Benchmark Heaven (−0.5) and games_bench (−0.9), both within run-to-run noise.

Results

All numbers measured with the same benchmark client; each model served by its own runtime (this repo's qev_fast engine, per-question requests). basal-omni-pico (Remek/basal-omni-pico) was served by basal 2.4.0, fast mode.

	squad-0.8b (previous)	squad-0.8b (this revision)	basal-omni-pico
Benchmark Heaven (public, 231)	77.1	76.6	75.8
JevExtra	71.3	71.7	64.6
JevSpatial	75.8	79.7	68.4
Rules-7	63.8	74.1	69.5
JevBench accuracy	84.9	86.4	84.0
JevBench coherence	70.6	76.4	74.2
KLEJ (Polish, 7 tasks, 500 test items each)	70.9 (trunk only)	73.4	75.3
games_bench (9 unseen games, closed loop; 0 = random, 100 = planner)	11.3	10.4	7.5
e2e on 8 real web apps (61 tests × 3 runs)	10 / 183	34 / 183	–

Speed on one NVIDIA L40S (Benchmark Heaven requests, one question each, after warm-up):

	this revision	basal-omni-pico (fast)	basal-omni-pico (eager)
single request, median / p95	3.7 / 30.5 ms	6.0 / 31.5 ms	50.4 / 101.3 ms
16 requests at once	115 decisions/s	94 decisions/s	33 decisions/s
GPU memory	6.3 GB	–	–

Read these honestly:

Benchmark Heaven moves by about ±1 point between runs (one item is 0.43 points); its hard tier (111 items) is around 57% for every model here.
Rules-7: part of the gain comes from procedurally generated rule-application data whose layout (the policy inside the question) resembles Rules-7's. No benchmark text was used; labels are computed by code. Sub-scores vary across training seeds.
The e2e expert is used only when the router gives it at least 35% (a serving setting chosen while looking at Benchmark Heaven and the real-app tests). It still fails the TodoMVC apps (it clicks instead of pressing Enter).
games_bench is our own closed-loop benchmark of games never trained on; all models are close to random when a game is shown as a raw grid, coordinates or prose.
Image input was not re-evaluated for this revision (the trunk changed; the vision tower is unchanged).
Benchmark scores were used for reporting; checkpoints were chosen on our own validation data, except the serving threshold above.

How it works

Trunk: a full fine-tune of Qwen3.5-0.8B-Base (24 layers, Gated DeltaNet + gated attention). It continues the previous qev-0.8b trunk on procedurally generated, exactly labelled data: consistency twins (the same decision asked in rephrased, reordered, merged or complementary forms, trained with a consistency loss), rule application with natural "up to / below" semantics, graded similarity, plus replay of the old data. Final weights are a weight average of three runs. No LLM-written text is in the trunk.
Experts: rank-64 LoRA on gate_proj, up_proj, down_proj of layers 12–23, y = W x + 2.0 · B A x, stored as 2-bit rows (specialists/slots_2bit.safetensors). Slot 1 games, 2 e2e, 3 multilingual, 4 documents. Trained by reinforcement learning only (closed-loop games, simulated web apps, decision tasks with a proper-scoring reward), trunk frozen. The multilingual and documents experts learned from data written and cross-checked by two LLMs.
Router: LayerNorm → Linear(1024, 256) → GELU → Linear(256, 9) on the mean RMS-normalised residual after layer 5 (specialists/router.safetensors), top-1 of {trunk, games, e2e, multilingual, documents}. Trained on measured outcomes (log-probability of the right answer) on validation rows no expert saw, so it only calls an expert where it helps. router.min_p_slots in the manifest holds per-expert confidence thresholds (e2e: 0.35).

Quickstart (NVIDIA GPU)

pip install -r requirements.txt -r requirements-cuda.txt

from qev import QEV

m = QEV.from_pretrained("labqoat/squad-0.8b", specialists=True)     # router + experts (fast CUDA engine)
m.decide(state, questions, per_question=True)                        # System One answers
m.last_routes                                                        # e.g. ["games"]

python qev_server.py --model labqoat/squad-0.8b --specialists     # POST http://127.0.0.1:8017/v1/systemone

Without specialists=True you get the trunk alone (not the old qev-0.8b: the trunk changed in this revision; pin the previous revision 2043fda for it). Experts need the fast CUDA engine (requirements-cuda.txt). The MLX and GGUF repos still hold the previous version and will be updated separately.

Files

model.safetensors (new trunk; embeddings, head and vision tower as before) and tokenizer / config files · specialists/ (manifest, router, 2-bit experts) · qev.py, qev_server.py, qev_fast/ (runner, server, CUDA engine with experts) · qev_image.py, qev_image_server.py (transformers image path, trunk only) · SHA256SUMS.

Limitations

Text decisions only were evaluated for this revision. The e2e expert is far from reliable on real apps (34 of 183 real-app test runs pass). Polish sentiment and QA (KLEJ) are behind basal-omni-pico. Probabilities are calibrated with the previous version's temperatures; check them on your own data before using them as error rates.

License: Apache-2.0.