---
license: mit
language:
- en
- zh
library_name: transformers
pipeline_tag: text-generation
---

![ZGCM-1 Logo](assets/zgcm1-logo.png)

# ZGCM-1-7B

**ZGCM-1-7B is a 7.39B-parameter dense language model trained from scratch**, built for mathematical reasoning and tool-assisted search. It combines deliberate internal thinking with active information gathering, supporting **256K-token context** and both thinking and direct-response modes in a single model.

## Highlights

- Hybrid gated sliding-window and global attention.
- 3.94× higher throughput and 6.4× lower KV-cache use than full attention at 256K context.
- FP8 + Muon + TWEO training recipe, with an estimated 4.16× pre-training time-to-loss improvement.
- Progressive 16K → 64K → 256K context training with mixed think/no-think post-training.

## Model specifications

| | |
|---|---|
| Parameters | 7.39B |
| Layers | 32 |
| Hidden size | 4096 |
| Intermediate size | 11008 |
| Attention | 32 query heads / 8 key-value heads |
| Attention pattern | 27 gated sliding-window layers + 5 global layers |
| Sliding-window size | 128 tokens |
| Vocabulary size | 155,136 |
| Context length | 262,144 tokens |
| Data type | BF16 |

The model uses custom Transformers code. Set `trust_remote_code=True` when loading it.

## Selected results

| Benchmark | Score |
|---|---:|
| MATH-500 | 97.13 |
| AIME 2026 | 75.00 |
| HumanEval+ | 90.24 |
| LiveCodeBench v6 | 46.86 |
| WebWalkerQA | 63.1 |
| Binary Function Search | 62.0 |

Scores follow the evaluation protocols described in the technical report.

## Quickstart

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zgcagi/ZGCM-1-7B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype="bfloat16",
    device_map="auto",
)

messages = [{"role": "user", "content": "Compute 1+1."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=4096)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

For vLLM, keep sliding-window attention enabled.

## WeChat Community

Scan the QR code to join the ZGCM-1 community group.

<p align="center">
  <a href="assets/zgcm-1-wechat-group-1.jpg">
    <img src="assets/zgcm-1-wechat-group-1.jpg" width="360" alt="ZGCM-1 WeChat Group QR code">
  </a>
</p>

## License

MIT License. See [LICENSE](https://huggingface.co/zgcagi/ZGCM-1-7B/blob/main/LICENSE).

## Citation

```bibtex
@misc{zgcm2026,
  title  = {ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search},
  author = {{ZGCM Team}},
  year   = {2026},
  url    = {https://github.com/zgcagi/zgcm-1}
}
```