---
license: apache-2.0
tags:
- math
- mathematics
- arithmetic
- tiny
- 10k
- smol
- R1
- think
- thinking
- reason
- reasoning
- mini
- TLM
- SLM
- math reasoning
- nanbeige
- huggingfaceTB
- axiomiclabs
- meta
- facebook
- huggingface
- small
language:
- en
pipeline_tag: text-generation
new_version: MoreThought/AresMath1.0-IT-GGUF
library_name: jax
metrics:
- accuracy
---
![Profile:](MoreThoughtProfile.webp)
# AresMath1.0-IT

**AresMath1.0-IT** (AresMath1.0-Iterative Tiny) is an 11,008 parameter TLM made to solve exact-match arithmetic problems even ones including ().

It uses a heavily **looped architecture** similar to that of nanbeige-4.2-3b to achieve superior mathematic accuracy on **unseen problems**.

It has 21 vocab, a 32 token context window, a 32 token output window, and 4 attention heads.

## Benchmarks

**AresMath1.0-IT** achieves comparable/exceeding performance against models **over 10,000x bigger** than it.

It exceeds almost every tested SLM's scores on arithmark 3.0 (converted into a format it can understand) while generating **exact-match** answers instead of guessing.

**AresMath1.0-IT**'s scores at different tempatures:

| Temp |  Accuracy  |   Total  |
|------|------------|----------|
| 1.00 | **42.30%** | 423/1000 |
| 0.90 | **44.70%** | 447/1000 |
| 0.80 | **47.10%** | 471/1000 |
| 0.70 | **48.90%** | 489/1000 |
| 0.60 | **50.20%** | 502/1000 |
| 0.50 | **52.40%** | 524/1000 |
| 0.40 | **52.80%** | 528/1000 |
| 0.30 | **53.80%** | 538/1000 |
| 0.20 | **54.70%** | 547/1000 |
| 0.10 | **54.80%** | 548/1000 |

Top **AresMath1.0-IT** score vs top SLM scores:
![Scores:](eval.png)

| Model |  Accuracy  |   Total  |
|------|------------|----------|
| MobileLLM-R1 | **65.70%** | 657/1000 |
| AresMath1.0-IT | **54.80%** | 548/1000 |
| GPT-X3-150M | **50.90%** | 509/1000 |
| OdinNext-138M | **41.80%** | 418/1000 |
| OdinNext-138M-Instruct | **40.90%** | 409/1000 |
| Quark-135M | **39.70%** | 397/1000 |
| SmolLM2-135M | **39.20%** | 392/1000 |
| SmolLM-135M | **36.80%** | 368/1000 |

## Training

It was trained on **13B** tokens of pretraining and **13B** tokens of RL, totalling **26B** tokens or **2,361,918** tokens per parameter.

The data was all **synthetically generated** and completely random, the problems were **not** made to be similar to arithmark 3.0 problems so it can score better.

It slowly climbed from around **30%** accuracy at 5B tokens to the current **50%** accuracy, jumping from negative RL rewards to +8.2 at the end.

It **seemed** like it hit ceilings many times in the training run but it **always** managed to slowly push past all of them.

## Training Hardware

This model was trained on multiple v5e TPUs.