LFM2.5-1.2B-Instruct ONNX - Vulkan / dual-role decoders
Vulkan-ready Instruct decoders for the LFM2.5 Audio + Instruct dual-role bundle (role-isolated sessions; the Instruct role handles text/tool I/O while the audio role handles speech). This repo holds only the modified Instruct artifacts - the audio q4 stack (decoder, encoder, detokenizer, vocoder, embeddings) is byte-identical to the official LiquidAI/LFM2.5-Audio-1.5B-ONNX release and is referenced from there, not re-hosted.
Files
| file | bytes | sha256 | notes |
|---|---|---|---|
| decoder_q4.onnx | 183,410 | d839af62f6c95e97d8cbf19839aabb5ce362a3f105fa7d4a75e9a624284848ef | primary - q4 decoder (external data: decoder_q4.onnx_data) |
| decoder_q4.onnx_data | 850,059,264 | d9666c44e2acc32f06c9351f9e7c4fd66bc060d88ca8e7f954e836c6845f7488 | q4 weights (MatMulNBits) + scales + zero points |
| decoder.onnx | 148,426 | 0c7c4b4ff0568f01f5e36a4106686981bd297675fae9b104116e71514e75bd57 | f32 fallback decoder (external data: decoder.onnx_data) |
| decoder.onnx_data | 4,714,127,360 | ba0b81edc781e59ceb88d44569359a5c49c0d6087498e4611ee901f681120303 | f32 weights (exact f16->f32 upcast of the Liquid fp16 export, weights swapped from the Instruct bf16 checkpoint) |
| embed_tokens.bin | 536,870,912 | ece8b635baf1918674073ee0900e782e6210f1d5b93173089ad0b6e07c6df139 | Instruct text embed table, f32 [65536, 2048] little-endian (exact bf16->f32 from the checkpoint) |
| embed_tokens.json | 96 | 2aaf7f0bcfb26496769d7cc5f50f095558bea297377c40eb2f8e357bae6f7d0c | embed table metadata |
| dual_role_manifest.json | - | - | bundle manifest (roles, contract, provenance) |
| instruct_q4_provenance.json | - | - | q4 surgery provenance (source pins, procedure, hashes, validation) |
Graph contract
Both decoders expose the same 24-input contract as the audio q4 decoder: inputs_embeds (f32, first), attention_mask (i64), and per-layer past_conv.* / past_key_values..{key,value} (f32). Outputs: logits + present_conv./ present.*.{key,value} (the q4 graph has no hidden_states output - the Instruct role reads logits only).
What was modified (vs the official Liquid Instruct ONNX release)
Source: LiquidAI/LFM2.5-1.2B-Instruct-ONNX @ commit 10f72e70abf67ac0fd7ebf15bc5854726891d864 (model_q4.onnx). The official release was NOT runnable as-is in the target runtime (onnx-vulkan-rs, a Vulkan ONNX interpreter); three fixes were applied - see instruct_q4_provenance.json for the full record:
- Embedding contract: the release takes input_ids (i64) and embeds internally via a custom q4 GatherBlockQuantized over a q4-quantized embed table. That node was removed and the input swapped to inputs_embeds (f32) - the runtime feeds embeddings from embed_tokens.bin (byte-identical to the audio q4 24-input contract). The weight-tied q4 lm_head (which reuses the embedding's scales + zero points) is preserved, so logits stay q4-quantized.
- Node order: the Liquid export node list was non-topological (110 violations); it was stably topo-sorted (ties on original index; 0 violations after).
- External-data locations repointed to decoder_q4.onnx_data.
The f32 fallback decoder derives from the same export family's fp16 graph (topo-sorted) with all 148 weight initializers swapped from the Instruct bf16 checkpoint (LiquidAI/LFM2.5-1.2B-Instruct @ 6314d2b7cf28a6ae9de9d3e77dcfcd9c9f281c77) and an exact f16->f32 upcast of the remaining 150 f16 initializers (the target engine is f32/i32/q4-only; it has no f16 compute path).
Quantization format note (q4)
The Instruct q4 export is asymmetric MatMulNBits: 93 nodes, bits=4, block_size=32, with a 4th input - a per-block packed-int4 zero point (uint8 tensor of shape [n][k/64], two int4 per byte, low nibble first). The Audio release is the 3-input symmetric form (zero point == 8). A runtime must implement the asymmetric form to use decoder_q4.onnx; the onnx-vulkan-rs fork that produced this bundle adds packed-int4 zero-point variants of its Q4 kernels for exactly this (fork: https://github.com/matbeedotcom/onnx-vulkan-rs-edge, commit 2d660f2). onnxruntime CPU runs the 4-input form natively.
Validation
- CPU (onnxruntime, greedy): the B4 tool-call prompt (88 ids) produces 21/21 tokens bit-identical to the bf16 PyTorch reference, including the tool_call_start / tool_call_end wrapper tokens: [get_weather(city="Paris")]Checking the current weather in Paris.
- Reproducibility: the committed export pipeline (dual_role/instruct_q4_export.py in the project repo) re-derives decoder_q4.onnx byte-identically (sha256 d839af62...).
License
LFM Open License v1.0 - inherited from the upstream Liquid releases (see LICENSE). The modifications are graph surgery + exact dtype conversion only; no weight values were retrained or requantized.