Instructions to use IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF # Run inference directly in the terminal: llama cli -hf IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF # Run inference directly in the terminal: llama cli -hf IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF # Run inference directly in the terminal: ./build/bin/llama cli -hf IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
Use Docker
docker model run hf.co/IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
- Llama new
- LM Studio
- Jan
- vLLM
How to use IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
- Ollama
How to use IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF with Ollama:
ollama run hf.co/IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
- Unsloth Desktop
- Pi
How to use IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF with Docker Model Runner:
docker model run hf.co/IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
- Lemonade
How to use IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
Run and chat with the model
lemonade run user.Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Axion-1.0 35B - MiniPlus V2.1
APEX Quantization Series by IsValorum
Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.
If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.
Vision: this repo ships the text GGUF. For multimodal use, grab
Axion-1.0-35B.mmproj-bf16.gguf(or its quantized versions) from the upstream GGUF repo.MTP/DSpark: the main GGUF has no MTP tensors and no DSpark draft is included for this family.
Family cross-links
| Tier | File | Base | Status |
|---|---|---|---|
| MiniPlus | Axion-1.0-35B.APEX-I-MiniPlus-V2.1.gguf |
Axion-1.0-35B | Published |
| NanoPlus | see repo | Axion-1.0-35B | Published |
Architecture and precision
- Base model:
Axion1-0/Axion-1.0-35B(Apache-2.0). BF16 GGUF reference from prithivMLmods/Axion-1.0-35B-GGUF. - Architecture: 40 layers, 30 linear-attention/SSM + 10 full-attention; 256 routed experts + 1 shared, 8 active; hidden 2048; 262,144-token context; 248,320 vocab.
- iMatrix: public
Axion-1.0-35B.imatrix.ggufby mradermacher, size and sha256 verified on download before use. - Recurrent shielding:
ssm_alpha,ssm_a,ssm_dt.bias,ssm_conv1dand RMSNorms remainF32. - Layer 0:
attn_gateinQ8_0; the linear-attention projections (attn_qkv,ssm_beta,ssm_out) inQ3_K.
Surgical Tensor Quantization Map (Audited from GGUF)
The exact tensor breakdown below has been verified directly from the compiled binary weights:
| Layer Group | Sub-Component / Tensor | Qty | Precision | Engineering Rationale |
|---|---|---|---|---|
| Global Output Head | output.weight |
1 | Q6_K |
Preserves near-FP16 token classification; stable syntax and bracket behavior. |
| Global Embeddings | token_embd.weight |
1 | Q4_K |
High-fidelity vocabulary representation. |
| All Normalizations | output_norm, attn_*_norm, post_attention_norm, ssm_norm |
131 | F32 |
Uncompressed numerical stability across all layers. |
| Expert Routers | blk.*.ffn_gate_inp, ffn_gate_inp_shexp |
80 | F32 |
Uncompressed routing over 256 experts; zero router drift. |
| Attention Gates | blk.*.attn_gate.weight (30 Hybrid Layers) |
30 | Q8_0 |
High-precision gating across the hybrid linear-attention layers. |
| Shared Foundation Experts | blk.*.ffn_{gate,down,up}_shexp (All 40 Layers) |
120 | Q5_K |
Active on every token; protected in high precision. |
| Periodic Full Attention | blk.{3,7,11,...}.attn_q/k/v (10 Anchor Layers) |
30 | Q4_K |
Full quadratic-attention anchors for deep needle-in-a-haystack retrieval. |
| Periodic Full Attention | blk.{3,7,11,...}.attn_output (10 Anchor Layers) |
10 | Q6_K |
Armored attention output projection over long context. |
| Recurrent SSM Scales | blk.*.ssm_alpha, ssm_a, ssm_conv1d, ssm_dt |
120 | F32 |
F32 shielding against recurrent-state drift. |
| Linear Attention & SSM | blk.*.attn_qkv, ssm_beta, ssm_out |
90 | Q3_K |
Linear-attention projections kept lean for CPU execution. |
| Edge MoE Experts | Layers 0-9 & 30-39 (ffn_*_exps) |
60 | Q3_K |
Routed experts calibrated with the importance matrix. |
| Core MoE Experts | Layers 10-29 (ffn_*_exps) |
60 | IQ3_XXS |
Deep-band routed experts calibrated with the importance matrix. |
The machine-readable version is tensor-map.tsv.
Files
| File | Size | Notes |
|---|---|---|
Axion-1.0-35B.APEX-I-MiniPlus-V2.1.gguf |
14,751,301,312 bytes | Main MiniPlus V2.1 GGUF |
| tensor-map.tsv | 33,146 bytes | Per-tensor types |
| chat_template.jinja | from source | Axion chat template |
Usage
llama-cli -m Axion-1.0-35B.APEX-I-MiniPlus-V2.1.gguf -p "Hello" -ngl 99 -c 32768
llama-server -m Axion-1.0-35B.APEX-I-MiniPlus-V2.1.gguf --jinja -ngl 99 -c 32768
Limitations
- Base benchmarks are self-reported by the Axion team; no independent evaluation is claimed for this tier.
- Quantization quality is theoretical until you test it on your own workload.
License
Apache-2.0, inherited from the base model. See the source model card.
- Downloads last month
- 765
We're not able to determine the quantization variants.
Model tree for IsValorum/Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF
Base model
Axion1-0/Axion-1.0-35B