KRAFTON

/

Raon-1-mini-preview

Like 226

Follow

KRAFTON 369

Thanks for releasing Raon-1 mini. I ran the preview (Q4_K_M GGUF, your llama.cpp branch fb89566) on a Korean RAG benchmark with known answers (Allganize RAG-Evaluation-Dataset-KO, 299 scorable questions over 64 Korean public PDFs). Every model got the same top-5 retrieved pages and the same instruction; answers were judged against the gold answers by an LLM judge (88.5% agreement with the O/X labels published with the dataset), plus a second judge from a different model family.

• Thinking on, temp 0: 74.2% (second judge 76.9%)
• Thinking on, model-card sampling (temp 1.0, top_p 0.95, top_k 20, presence 1.5): 74.9% (76.6%)
• Thinking off: 72.6% (74.9%)
• For reference on the same setup: qwen3.8 27B thinking 77.3%, thinking off 71.2%; EXAONE 3.5 7.8B 63.2%

One thing that may be useful to you: with thinking on at temperature 0, 16 of 320 requests used the whole 8,192-token budget in reasoning and returned an empty answer. With the recommended sampling it was 11 of 320 (single run each). qwen3.8 27B hit no budget limit on the same prompts.

It also builds and runs on 2x GTX 1080 Ti (sm_61): ~10 tok/s generation vs ~96 tok/s on an RTX 3090.

Full write-up with build commands, per-question verdicts and scripts: https://bric.pe.kr/blog/krafton-raon-1-mini-korean-rag-eval-rtx3090-gtx1080ti (Korean version: https://bric.pe.kr/blog/krafton-raon-1-mini-korean-rag-300-questions-local)