Harsha001/Underdog-Bark-3.0-XDNA2

Underdog Bark 3.0 for the AMD Ryzen AI NPU (XDNA2)

Built from ConwayResearch/Underdog-Bark-3.0, which is derived from NVIDIA's Parakeet TDT 0.6B v3 and Moondream's parakeet-ultra.

English speech recognition with the encoder running on the NPU of AMD Ryzen AI 300 and Ryzen AI MAX PCs (XDNA2, for example Strix Point and Strix Halo), through Windows ML. No AMD account, no Ryzen AI SDK, no PyTorch: pip install, and Windows downloads the NPU provider itself.

The download is about 170 MB. This repo ships Conway Research's own 163 MB model.bark file unchanged. On the first run the runtime unpacks it into the full-precision encoder weights (2.44 GB, kept next to the model) and checks the result against a SHA-256, so the weights on your NPU are exactly the weights measured below.

• The encoder (subsampling, 24 Conformer blocks, projection) is one static-shape ONNX graph per input length (10 s, 20 s and 36 s). The NPU compiler turns it into BF16 and runs it on the NPU.
• Feature extraction, the small TDT decoder (two LSTM layers and the joint network) and the greedy search run on the CPU in plain NumPy.
• Like Bark's own runtimes, 0.25 s of silence is put before every recording. Longer audio is cut into pieces of up to 36 s at the quietest moment near each cut.

Accuracy

LibriSpeech test sets from the Open ASR Leaderboard (hf-audio/esb-datasets-test-only-sorted), every utterance (2,620 and 2,939), scored with the leaderboard's English text normalizer and corpus word error rate (lower is better).

	test-clean	test-other
This repo, encoder on the NPU (BF16)	2.31 %	4.60 %
Same pipeline, full precision on the CPU (Conway's own bark_loader.py + transformers)	2.31 %	4.63 %
Conway Research's published figures for Bark 3.0	1.89 %	4.08 %

The NPU matches the full-precision reference on both sets. Conway publishes lower numbers from its own scoring setup; our full-precision reference, measured with the same scripts as the NPU row, sits about 0.4 to 0.5 points above them (we saw the same gap for Phonon-2). Compare the NPU against the reference row, not across setups.

Speed

Measured on a Ryzen AI MAX+ 395 (Strix Halo, 50 TOPS XDNA2 NPU, 16 Zen 5 cores), steady state, encoder only:

Window	NPU (BF16)	CPU, 16 cores (FP32, ONNX Runtime)
10 s	473 ms (21x realtime)	243 ms (41x realtime)
20 s	751 ms (27x realtime)	463 ms (43x realtime)
36 s	2,028 ms (18x realtime)	834 ms (43x realtime)

Over the full test sets, end to end (features, encoder, decoder, search), the released runtime ran at about 11x realtime on the NPU.

What the NPU is for: on this chip the CPU is faster for a single stream. The NPU's value is that it leaves the CPU and GPU free for other work and is built to use less power (not measured here), which matters for always-on dictation and background transcription on a laptop.

Quick start

Windows 11 on a Ryzen AI 300 / AI MAX PC, Python 3.12, an up-to-date NPU driver.

py -3.12 -m venv .venv; .venv\Scripts\activate
pip install huggingface_hub
hf download Harsha001/Underdog-Bark-3.0-XDNA2 --local-dir Underdog-Bark-3.0-XDNA2
cd Underdog-Bark-3.0-XDNA2
pip install -r requirements.txt
python bark_xdna2.py --prepare                 # unpacks the weights and compiles the three lengths, once (about 2.5 hours)
python bark_xdna2.py recording.wav             # any sample rate; mono or stereo
python bark_xdna2.py recording.wav --device cpu

Without --prepare the weights unpack on the first run (about a minute) and each length compiles the first time audio of that length shows up (about 45 minutes each on a Ryzen AI MAX+ 395). The compiled encoders go to npu_cache/ (about 0.8 GB each) and load in seconds afterwards.

From Python:

from bark_xdna2 import Bark, load_audio
asr = Bark(".", device="npu")
print(asr.transcribe(load_audio("recording.wav")))

Files

File
model.bark	Conway Research's packed Bark 3.0 weights, byte for byte as published (revision 85db2ad)
bark_loader.py, NOTICE-Underdog-Bark-3.0	Conway's exact decoder for model.bark and its notice, unchanged
encoder_layout.json, encoder_extra.bin	where each encoder tensor sits in encoder.weights; the 1 MB of folded convolution and normalization weights that are not plain copies
encoder_T1000.onnx, encoder_T2000.onnx, encoder_T3600.onnx	the encoder for 10 s, 20 s and 36 s of audio (opset 18), reading encoder.weights
mel_filters.npy, vocab.json	128-band mel filter bank and the 8,192-piece vocabulary with the 5 TDT durations
bark_xdna2.py, requirements.txt	the runtime: unpacking, features, NPU setup, chunking, decoder, TDT greedy search
build/	the scripts that made these files and measured them

Made on first run, not downloaded: encoder.weights (2.44 GB), decoder_weights.npz (the decoder and joint network, about 70 MB) and npu_cache/.

How this was made

1. model.bark (revision 85db2ad) was decoded with Conway's own bark_loader.py and loaded into transformers' ParakeetForTDT. That is the full-precision reference in the table above.
2. The encoder was rewritten for a static-shape NPU graph (build/bark_npu.py): padding masks for the three subsampling stages, the convolution module and attention come in as input tensors, and attention uses a finite additive mask (-10,000) so BF16 never meets infinity. On the CPU in ONNX it matches the transformers encoder to about 1e-5 (relative 3e-6) at all three lengths.
3. Exported with torch.onnx (dynamo, opset 18) for 1,000 / 2,000 / 3,600 feature frames, then rewritten to read one shared weight file (build/share_weights.py). build/pack_release.py records how that file is rebuilt from model.bark: 566 tensors are plain or transposed copies, 48 folded tensors ship in encoder_extra.bin.
4. Compiled by the VitisAI execution provider that Windows ML installs (MicrosoftCorporationII.WinML.AMD.NPU.EP 1.8.75), which converts the FP32 graph to BF16 for the NPU.
5. Measured with build/eval_wer.py (accuracy) and build/bench.py (speed). The released runtime itself was then run from these exact files, compiled from scratch, on both test sets on the NPU: 2.32 % and 4.59 % WER, with 96 % and 94 % of transcripts word for word identical to the full-precision reference (the rest differ by a word here and there, in both directions).

A fix you may need elsewhere: the NPU compiler cannot resolve file paths inside C:\Program Files\WindowsApps (Windows gives no short names there), so a first compile fails with Failed to load VFS from C:\PROGRA~1\WindowsApps\. The runtime loads the provider through a directory junction in npu_cache/ep/, which avoids it.

Limits

• Made for Windows 11 with an XDNA2 NPU (Ryzen AI 300 and AI MAX series) and tested on a Ryzen AI MAX+ 395 only. The --device cpu path runs anywhere ONNX Runtime does.
• English only, like Bark 3.0. One stream at a time; no batching and no word timestamps in this runtime.
• The first run needs about 2.5 GB of free disk for the unpacked weights, and each first compile needs several GB of free memory.

Licence and credits

The model files are a derivative of Underdog Bark 3.0, Parakeet TDT 0.6B v3 and parakeet-ultra and are distributed under CC-BY-4.0 (LICENSE-WEIGHTS-CC-BY-4.0.txt), the same as all three.

• Underdog Bark 3.0 © Conway Research, CC-BY-4.0; its NOTICE is included as NOTICE-Underdog-Bark-3.0.
• Parakeet TDT 0.6B v3 © NVIDIA Corporation, CC-BY-4.0. parakeet-ultra © Moondream, CC-BY-4.0.
• Changes made here: no retraining. The encoder was re-expressed as three static-shape ONNX graphs with explicit masks, its depthwise convolution and batch normalization were folded together, and the feature extraction, decoder and greedy search were re-implemented in NumPy. On the NPU the encoder runs in BF16. Not made, sponsored or endorsed by Conway Research, Moondream, NVIDIA or AMD.
• The runtime and build scripts in this repo are Apache-2.0 (LICENSE-CODE-Apache-2.0.txt), except bark_loader.py, which is Conway Research's file, published with Bark 3.0 under CC-BY-4.0 and included unchanged.