How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("automatic-speech-recognition", model="chienhsinlin/helioslm-asr-tw-v6m")
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq

processor = AutoProcessor.from_pretrained("chienhsinlin/helioslm-asr-tw-v6m")
model = AutoModelForSpeechSeq2Seq.from_pretrained("chienhsinlin/helioslm-asr-tw-v6m", device_map="auto")
Quick Links

羲和 HeliosLM ASR TW v6m

羲和(HeliosLM)ASR TW v6m 是以 OpenAI Whisper large-v3 為基礎、針對台灣華語(繁體中文輸出)與台語(漢字輸出)微調的語音辨識模型。它是羲和對話模型 helioslm-chat-e4b 的語音搭配模型。

HeliosLM ASR TW v6m is a speech recognition model fine-tuned from openai/whisper-large-v3 for Taiwanese Mandarin (Traditional Chinese output) and Taiwanese Hokkien (Han-character output).

v6m 加強台語:台語漢字字錯率較 v5 低,華語略高。以華語為主請用 v5,台語為主請用 v6m。 v6m improves Taiwanese Hokkien over v5; Mandarin is slightly worse. Use v5 for mostly-Mandarin audio, v6m for Taiwanese Hokkien.

  • 參數量 Parameters: about 1.55B (Whisper large-v3 architecture)
  • 授權 License: Apache-2.0 (same as the base model)

訓練資料 Training data

All public data, no private recordings:

  • Mozilla Common Voice 27.0 zh-TW and nan-tw (CC0-1.0), test sentences and test speakers excluded
  • Google FLEURS cmn_hans_cn train/dev (CC-BY-4.0), transcripts converted to Traditional Chinese with OpenCC
  • AISHELL-1 train subset (Apache-2.0)

Stage 1 (= v4): 1 epoch, learning rate 7e-6, 4,776 steps, then merged with the base model (0.85 × fine-tuned + 0.15 × whisper-large-v3) to give v5.

Stage 2 (v6m): continued from v5 with Common Voice nan-tw (Han-character transcripts) upsampled 4×, 30% of the Common Voice zh-TW clips and FLEURS 2×; learning rate 4e-6, about 2.2 hours; then merged again (0.85 × stage 2 + 0.15 × whisper-large-v3).

評測 Evaluation (2026-11-05/06, self-run)

Character error rate (CER, %, lower is better), greedy decoding, same clips and same run for both models. Scores are self-reported.

Test set Clips HeliosLM ASR TW v6m HeliosLM ASR TW v5 whisper-large-v3 (base)
Common Voice 27.0 zh-TW test 998 5.42 5.10 15.25
FLEURS cmn test (Traditional Chinese refs) 945 7.61 7.33 7.54
Common Voice 27.0 nan-tw test, Han characters 1,000 32.69 38.15 231.13
Common Voice 27.0 nan-tw test, raw transcripts (incl. romanization) 1,000 85.09 86.29 121.57

Taiwanese Hokkien is still weak: the model writes Han characters only and does not output Tâi-lô romanization.

使用 Usage

from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="chienhsinlin/helioslm-asr-tw-v6m", chunk_length_s=30)
print(asr("audio.wav", generate_kwargs={"language": "zh", "task": "transcribe"})["text"])

模型可能辨識錯誤,重要內容請自行確認。The model can make mistakes; verify important transcripts.

Downloads last month
52
Safetensors
Model size
2B params
Tensor type
F16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for chienhsinlin/helioslm-asr-tw-v6m

Finetuned
(1,109)
this model

Collection including chienhsinlin/helioslm-asr-tw-v6m

Article mentioning chienhsinlin/helioslm-asr-tw-v6m