Spestly/Athena-4-15B

Overview

Athena-4-15B is a 15-billion-parameter multimodal reasoning model designed for high-quality textual reasoning and image understanding while remaining memory-efficient enough to run on a single modern GPU. The design and training approach are informed by the Apriel-1.5-15b-Thinker research and implementation (mid-training + text SFT emphasis).

---

Key capabilities

• Strong textual reasoning (math, logic, chain-of-thought style outputs).
• Multimodal understanding: able to process image+text prompts for captioning and image reasoning via an image-text processor.
• Optimised for instruction-following use cases (SFT on curated instruction data).

---

Highlights / Benchmark notes

• Competitive performance on reasoning and multimodal benchmarks reported by the Apriel team (reported scores, e.g., Artificial Analysis index and IFBench in their model card). ([Hugging Face][1])
• Targeted to deliver high capability per parameter (aiming for frontier-level reasoning while keeping model size ~15B).

---

Intended uses

• Conversational assistants that require explicit stepwise reasoning.
• Question answering and knowledge retrieval where traceable, stepwise reasoning is valuable.
• Multimodal tasks requiring captioning, image understanding, or image+text reasoning.
• Research and internal tooling for fine-grained reasoning and benchmark comparisons.

Not intended for

• High-risk medical, legal, or safety-critical decision-making without human review.
• Any deployment that requires guaranteed factual accuracy without an external verification pipeline.

---

Limitations

• Generates internal chain-of-thought-style reasoning before final answer by design; this can increase token usage and latency. The Apriel upstream notes that the model explicitly produces stepwise reasoning and then a final response. This behaviour may need post-processing or filtering depending on your deployment.
• The model was trained and fine-tuned on curated datasets prioritising reasoning; domain coverage should be validated for specialised domains (medical, legal, etc.).

---

Safety & Responsible Use

• Use human-in-the-loop review for high-stakes outputs.
• Apply content filtering, rate limits, and prompt-based guardrails before public-facing deployment.
• Monitor for privacy-sensitive data leakage during fine-tuning or deployment and redact or avoid storing sensitive input data.

---

Training summary (reference implementation)

• Mid-training / continual pretraining: Extensive CPT on reasoning-focused text and multimodal interleaved image-text corpora to strengthen reasoning capabilities.
• Supervised fine-tuning (SFT): Fine-tuned on >2M high-quality text samples consisting of mathematical problems, coding tasks, instruction-following data, and conversational examples. No RLHF was applied in the referenced Apriel workflow.
• Training hardware (reference): Apriel reports large-scale training hardware usage (e.g., H100 clusters) in their public card; Athena’s training choices may differ but were informed by this regimen.

---

Evaluation

• Third-party and open-benchmark evaluations were used in the Apriel reference (Artificial Analysis for text benchmarks; VLMEvalKit/OpenCompass for image evaluation). Reported scores indicated strong reasoning performance relative to model size. Use case-specific evaluation is recommended before production deployment.

---

How to run (example)

Below is a minimal example inspired by the Apriel reference implementation. Adapt tokenizer/processor and device mapping for your runtime.

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Spestly/Athena-4-15B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

Note: Athena can be adapted to vLLM or other high-throughput inference backends. If you adopt a chain-of-thought style generation, add post-processing to extract the final answer boundaries as required.

---

License

Use a permissive license consistent with your organisation’s policy. The Apriel reference model uses an MIT license — check and align Athena’s license to your legal requirements before publishing.

---

Citation

If you publish results using Athena, include a citation to the design and training methodology foundation (the Apriel-1.5-15b-Thinker technical report and model card) and your own technical report describing Athena’s differences, datasets, and evaluation methodology. ([Hugging Face][1])

---

Implementation notes & recommendations

• Prompting: Athena benefits from prompts that ask for stepwise reasoning when the trace is required, but for concise outputs prefer instructing the model to “Answer concisely” or to “Provide only the final answer.”
• Latency vs. accuracy: Expect higher token usage and slightly longer generation time due to explicit internal reasoning; benchmark inference cost and consider temperature/top-k adjustments for production.
• Safety pipeline: Add toxicity checks, hallucination detection, and a facts-verification layer for external claims before surfacing to end users.
• Evaluation: Run domain-specific benchmarks and human evaluations for calibration prior to public release.