Whisper large-v3-turbo — Persian (Farsi)
A full fine-tune of openai/whisper-large-v3-turbo for
Persian speech recognition, trained on ~236 hours of openly licensed Persian speech: conversational YouTube
audio with creator-made subtitles plus formal read speech.
- Verbatim, colloquial-aware output. It writes what was said («نمیدونم», «میشه», «داره»), instead of rewriting speech into formal written Persian.
- Punctuated output («،», «.», «؟»).
- Fast: same size and speed as turbo (809 M parameters). A 70-minute podcast transcribes in about 25 s on an RTX 4070 Ti.
- A
whisper.cpp/ GGML version is included (ggml-persian-v3-turbo.bin), which also works with FFmpeg'swhisperfilter.
Results
Word error rate (%, lower is better). The test sets are fixed subsets, and the same clips were used for every model: 300 FLEURS clips, 150 + 150 held-out YouTube / YODAS2 segments, and 300 Neyshekar clips. Text is normalized before scoring: punctuation and diacritics are removed, Arabic/Persian letter and digit variants are unified, and spacing variants such as «می شه» / «میشه» / «میشه» are treated as the same word.
| Model | FLEURS | YouTube (unseen channels) | YODAS2 | Neyshekar | Mean |
|---|---|---|---|---|---|
| this model | 6.4 | 14.4 | 23.0 | 6.0* | 12.4 |
| MohammadKhosravi/whisper-large-v3-Persian | 11.9 | 30.8 | 34.8 | 12.9 | 22.6 |
| AmirMohseni/whisper-large-v3-persian-bf16 | 12.2 | 29.9 | 34.9 | 13.2 | 22.5 |
| sadrasa97/whisper-large-v3-turbo-fa | 14.4 | 34.5 | 41.2 | 16.6 | 26.7 |
| openai/whisper-large-v3 | 17.3 | 35.8 | 37.3 | 21.0 | 27.8 |
| openai/whisper-large-v3-turbo | 18.7 | 34.0 | 36.3 | 26.4 | 28.8 |
| nezamisafa/whisper-persian-v4 | 12.6 | 45.0 | 45.7 | 17.5 | 30.2 |
| vhdm/whisper-large-fa-v1 | 14.0 | 51.3 | 62.6 | 21.2 | 37.3 |
* This model was trained on Neyshekar's train split, which shares no speakers with its test split. That makes its Neyshekar score in-domain; the other models have not seen Neyshekar. It was also trained on FLEURS-train and on YouTube subtitles in the same style as the YouTube references, so read the FLEURS and YouTube columns with that in mind.
Unseen long-form test: a 70-minute episode of Bplus Podcast (not in any training data, manual subtitles). Each model is listed with the better of two long-form methods: pause-based pieces, with or without 2 s overlap (see Long recordings below).
| Model | WER | CER |
|---|---|---|
| this model | 11.2 % | 2.7 % |
| openai/whisper-large-v3-turbo | 19.6 % | 5.5 % |
| nezamisafa/whisper-persian-v4 | 32.4 % | 10.8 % |
Models that rewrite colloquial speech into formal Persian score worse on the YouTube sets, because the references are verbatim subtitles. That is a real difference in behaviour, not only in spelling.
Usage
Transformers
import torch
from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="givia/whisper-large-v3-turbo-persian",
dtype=torch.float16, device="cuda:0")
print(asr("audio.mp3", generate_kwargs={"language": "persian", "task": "transcribe"})["text"])
Long recordings
The pipeline call above handles one 30 s window. For longer audio, this approach gave the best results:
- Cut the audio at pauses into pieces of up to about 26 s, choosing the longest pause in each window.
- Decode each piece with 2 s of the neighbouring audio on both sides, plus 0.25 s of silence padding, so words at the edges are heard in context.
- Join neighbouring transcripts at the point where they agree on the same words, so nothing is duplicated.
On the podcast above, this lowered the error rate on words at piece edges from 24 % to 12 % (about the same as mid-piece words), and overall WER from 12.1 % to 11.2 %. Plain 30 s windows without overlap can duplicate or fuse words at the boundaries.
whisper.cpp
whisper-cli -m ggml-persian-v3-turbo.bin -l fa -osrt -f audio.wav
FFmpeg (8.0+, built with --enable-whisper)
ffmpeg -i video.mp4 -vn -af "whisper=model=ggml-persian-v3-turbo.bin:language=fa:queue=20:\
vad_model=ggml-silero-v5.1.2.bin:destination=video.srt:format=srt" -f null -
ggml-silero-v5.1.2.bin is whisper.cpp's voice-activity model (models/download-vad-model.sh silero-v5.1.2).
The GGML file is fp16; quantize it with whisper.cpp's quantize tool if you need a smaller file.
Training data (~236 h)
| Source | Hours | Type | License |
|---|---|---|---|
| Neyshekar (train split) | 95.6 | read speech, 134 speakers | CC0-1.0 |
| YouTube videos with creator-uploaded Persian subtitles (94 channels, 289 videos) | 55.0 | conversational, podcasts, lectures | CC-BY (YouTube Creative Commons), see attribution below |
| Common Voice 26.0 fa (curated train split) | 34.3 | read sentences, 214 speakers | CC0-1.0 |
YODAS2 fa000 (131 videos, manual subtitles) |
24.2 | conversational | CC-BY-3.0 |
| ManaTTS (~15 h subset) | 15.2 | formal read speech, 1 speaker | CC0-1.0 |
FLEURS fa_ir train |
12.1 | read speech | CC-BY-4.0 |
Processing:
- YouTube data: only creator-uploaded subtitles were used, never YouTube's auto-generated captions. Videos had
to carry a Creative Commons license at download time. Segments of up to 30 s were cut at subtitle boundaries and
kept only if
whisper-large-v3-turboroughly agreed with the subtitle (CER ≤ 0.4). Each channel was capped at 5 h. - Punctuation: subtitles without punctuation were punctuated by an LLM (Claude). Each result was accepted only if the words were unchanged once punctuation was removed; otherwise the segment was dropped.
- Short clips: clips from the read-speech corpora were packed per speaker into examples of up to 28 s.
- Labels: Arabic/Persian letter variants were unified, and bracketed sound tags (e.g. «(موسیقی)») were removed.
Training procedure
Full fine-tune of all 809 M parameters on a single RTX 4070 Ti (12 GB):
- 8-bit AdamW (bitsandbytes), learning rate 1e-5, 200 warmup steps, linear decay.
- bf16 autocast, gradient checkpointing.
- SpecAugment (5 % time and frequency masking).
- Effective batch size 16, 2 epochs (4,648 steps, about 4 h).
- The checkpoint was selected by WER on a held-out dev set: FLEURS-dev plus held-out YouTube channels.
Limitations
- Hard audio: noisy, overlapping or very fast speech is harder (about 23 % WER on YODAS2). Rare names and literary or technical vocabulary are the most common errors.
- Music: on long stretches of music without speech, Whisper models can produce a word such as «موسیقی». Using a voice-activity detector (as in the FFmpeg example) avoids this.
- Dialects: trained and evaluated on Iranian Persian. Dari and Tajik have not been tested.
- Numbers: written as digits or words inconsistently, following the mix of conventions in the training text.
- Not streaming: like all Whisper models, it transcribes chunks of audio, not a continuous stream.
Attribution
This model was trained on CC-BY-licensed material. We credit:
- FLEURS (Google, CC-BY-4.0)
- YODAS2 (ESPnet / CMU, CC-BY-3.0, and the creators of its videos)
- The creators of the following YouTube channels, whose Creative Commons videos and subtitles were used (channel — videos used — hours of training audio)
If you are a rights holder and believe material was used incorrectly, please open a discussion on this repository.
YouTube channels (94)
- Downloads last month
- 5,527
Model tree for givia/whisper-large-v3-turbo-persian
Base model
openai/whisper-large-v3Datasets used to train givia/whisper-large-v3-turbo-persian
Space using givia/whisper-large-v3-turbo-persian 1
Evaluation results
- WER (%) on FLEURS (fa_ir, test, 300-clip subset)test set self-reported6.400