--- library_name: mlx license: other license_name: lfm1.0 license_link: LICENSE pipeline_tag: audio-text-to-text tags: - liquid - lfm3 - edge - decision - classification - calibration - system-one - multimodal - vision - audio - mlx language: - en - ar - zh - fr - de - it - ja - ko - pt - es - vi - th - id - hi - ru - pl base_model: LiquidAI/d1-Omni-2B base_model_relation: quantized ---

API Try Docs Blog

# d1-Omni-2B-MLX-8bit 8-bit MLX version of [d1-Omni-2B](https://huggingface.co/LiquidAI/d1-Omni-2B) for Apple Silicon. The weights take 2.71 GB. d1-Omni-2B is a decision model for text, pictures, video, and audio. It answers named, typed questions over a state with calibrated probabilities in one forward pass, without generating tokens. Questions can be written or spoken. The language model uses 8-bit affine quantization with groups of 64. The vision tower, audio encoder, projectors and norms stay in FP16. Against the original checkpoint run in float32, it kept all 59 test answers and moved probabilities by at most 0.008. Find architecture and usage details in the [original model card](https://huggingface.co/LiquidAI/d1-Omni-2B). ## Use with MLX-VLM

Requires [MLX-VLM](https://github.com/Blaizzy/mlx-vlm) 0.7.11 or later: ```bash pip install -U "mlx-vlm>=0.7.11" ``` d1-Omni-2B answers questions without generating text, so chat apps and `mlx_vlm.generate` cannot run it. Use `model.system_one` or the `decide` command below. ```python import numpy as np from mlx_vlm import load from mlx_vlm.utils import load_image model, processor = load("LiquidAI/d1-Omni-2B-MLX-8bit") # Text: several named questions over one state, answered in one pass questions = { "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"}, "team": {"type": "choice", "instructions": "Which team should handle this?", "criteria": {"billing": "Charges, refunds, invoices", "shipping": "Delivery and parcels", "technical": "App or site faults"}}, "topics": {"type": "choice", "multiple": True, "instructions": "Which topics does the message raise?", "criteria": {"billing": "Payments and charges", "delivery": "Shipping and parcels", "login": "Signing in to the account"}}, "urgency": {"type": "score", "instructions": "How urgent is this ticket?", "criteria": ["low", "medium", "high"]}, } out = model.system_one("I was charged twice and my parcel never arrived. Please refund one charge.", questions) a = out["answers"] print(round(a["refund"]["noul"], 2), a["team"]["choice"], a["topics"]["choices"], a["urgency"]["legend"][str(int(a["urgency"]["score"]))]) # 0.94 billing ['billing', 'delivery'] medium # Picture: the photo is the whole state image = load_image("http://images.cocodataset.org/val2017/000000039769.jpg") out = model.system_one(None, { "cats": {"type": "number", "instructions": "How many cats are there?", "range": [0, 10], "step": 1}, "remote": {"type": "box", "instructions": "Where is the remote control on the left?"}, }, media=[image]) a = out["answers"] print(a["cats"]["number"], {k: round(v, 2) for k, v in a["remote"]["box"].items()}) # 2.0 {'top': 0.15, 'bottom': 0.24, 'left': 0.06, 'right': 0.27} # Audio: three short beeps over quiet noise, 16 kHz mono rate = 16000 clip = np.random.default_rng(0).normal(0, 0.003, 4 * rate).astype(np.float32) for start in (0.5, 1.5, 2.5): clip[int(start * rate):int(start * rate) + 3200] += 0.3 * np.sin(2 * np.pi * 880 * np.arange(3200) / rate) out = model.system_one(None, { "beeps": {"type": "number", "instructions": "How many beeps are there?", "range": [0, 10], "step": 1}, "speech": {"type": "noul", "instructions": "Does someone speak in the recording?"}, }, media=[{"type": "audio", "audio": clip, "sampling_rate": rate}]) a = out["answers"] print(a["beeps"]["number"], a["speech"]["noul"] > 0.5) # 3.0 False ``` `model.system_one` returns `{"answers": {name: answer}, "usage": {...}}`, and `model.system_one_batch` takes several requests. The original model card describes every question type and answer field. From the command line, the answers print as JSON: ```bash python -m mlx_vlm decide --model LiquidAI/d1-Omni-2B-MLX-8bit \ --state "I was charged twice this month. Please refund one of the charges." \ --questions '{"refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"}}' ``` More options in the [MLX guide](https://docs.liquid.ai/deployment/on-device/mlx).