Spaces:

---

AxiomicLabs

/

Open_SLM_Leaderboard

Like 129

Running

Adds Pollock 1.5, a 127,565,312-parameter model pretrained from scratch by Fabryka AI / SlayerLab.

Model: https://huggingface.co/SlayerLab/pollock-mini-lm-125m
Immutable revision: 4130d9854aaeb6940cae8729dcf3a8218a9eae9c
Weights SHA-256: 4a88432225dac6255ae1e2a085019f36dcdaf55fa33c77629dba6e7445c6c2fb

This PR adds the Fabryka AI organization entry (linked to /SlayerLab) and one model row to index.html.

Results are zero-shot acc_norm on full datasets in float32. lm-evaluation-harness 0.4.12 was used for the four standard tasks; ArithMark-3 used the official script pinned at revision 6f6e59dd9b7e2c63455f7af7f838f9ecc3d0a746.

Intelligence Index: 12.5885.

Reproduce the standard tasks with:

python -m lm_eval --model hf \
  --model_args pretrained=SlayerLab/pollock-mini-lm-125m,revision=4130d9854aaeb6940cae8729dcf3a8218a9eae9c,dtype=float32,max_length=2048 \
  --tasks hellaswag,arc_easy,arc_challenge,piqa \
  --num_fewshot 0 --batch_size 8 --device cuda:0 \
  --seed 0,1234,1234,1234

ArithMark-3 was run with the official bencharithmark-3.py using dtype float32, batch size 32, max context 1024, and acc_norm as the primary metric. The model uses stock GPT2LMHeadModel; trust_remote_code is not required.

Rules: 127,565,312 parameters (RULE 01); trained from random initialization by SlayerLab, with the full training record linked from the model card (RULE 02); public, ungated genuine training checkpoint and not a merge (RULE 03); listed zero-shot evaluations supplied above (RULE 04). The model card documents the native-to-Transformers conversion with maximum absolute logit error 0.0.