Ghana Fast Voice: base model

A ZipVoice model (123M parameters) fine-tuned on 542 hours of speech in 134 African languages plus Ghanaian English. It is meant to be a starting point for fine-tuning: language-specific or single-voice models, such as ghanaopenai/ghana-fast-voice, are trained from it. It is not packaged for deployment.

  • Architecture: ZipVoice (flow matching, Zipformer text encoder and decoder, 24 kHz Vocos mel features), fine-tuned from k2-fsa's pre-trained ZipVoice. Zero-shot voice cloning from a speech prompt is kept.
  • Text input: africa-g2p's universal orthography: lowercase a-z and spaces. Native text in any of the languages is converted to this one orthography with afrig2p, so one character tokenizer (tokens.txt, ZipVoice's simple tokenizer) covers every language. There are no language tags or speaker IDs.
  • Training data: ghanaopenai/african-speech-universal-tts: 258,205 clips (542.1 h) from every AfriSpeech-Selector subset, plus Ghanaian English podcast speech. The text is AfriSpeech/africa-whisper-ct2's universal-orthography transcript of each clip.
  • Training: about 6 epochs at a fixed learning rate (1e-4, ScaledAdam, bf16, one H200). It was early-stopped on dev loss, and this checkpoint averages epochs 3 to 5.
Dev loss (1,406 held-out clips)
k2-fsa ZipVoice, before fine-tuning 0.0711
after epoch 1 0.0632
after epoch 3 0.0620
after epoch 5 (best) 0.0617

Files

  • model.pt: the model weights ({"model": state_dict}, fp32)
  • model.json: model configuration
  • tokens.txt: the character vocabulary

Fine-tuning

Pass this checkpoint as --checkpoint, and tokens.txt as --token-file, to ZipVoice's train_zipvoice.py --finetune 1 --tokenizer simple. The training text must be in the universal orthography. Recipes for per-language and single-voice (no-prompt) fine-tuning are in GhanaOpenAI/ghana-fast-voice.

Zero-shot inference

python -m zipvoice.bin.infer_zipvoice --model-name zipvoice \
  --model-dir ghana-fast-voice-base --checkpoint-name model.pt --tokenizer simple \
  --prompt-wav prompt.wav --prompt-text "universal-orthography transcript of the prompt" \
  --text "universal-orthography text to say" --res-wav-path out.wav

License

CC BY-NC 4.0 (non-commercial), following the training data. ZipVoice's code and its original pre-trained weights are Apache-2.0.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ghanaopenai/ghana-fast-voice-base

Base model

k2-fsa/ZipVoice
Finetuned
(6)
this model
Finetunes
1 model