Update to 2607 version

Signed-off-by: Jason <jasoli@nvidia.com>

README.md

@@ -4,11 +4,30 @@ emoji: 🐠
 colorFrom: green
 colorTo: green
 sdk: gradio
-sdk_version: 6.9.0
-python_version: 3.11
+sdk_version: 6.19.0
 app_file: app.py
 pinned: false
 license: other
+license_name: nvidia-open-model-license
+license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license
+short_description: Nvidia Text-to-Speech with MagpieTTS.
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# MagpieTTS Multilingual Demo
+
+A live [Gradio](https://www.gradio.app/) demo for NVIDIA's **MagpieTTS Multilingual 357M** text-to-speech model. Type text, pick a language and speaker, and synthesize speech across 12 languages. The full model description and usage notes are shown inside the app.
+
+- 🤗 **Model weights:** [nvidia/magpie_tts_multilingual_357m](https://huggingface.co/nvidia/magpie_tts_multilingual_357m)
+- 🤗 **NanoCodec weights:** [nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps](https://huggingface.co/nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps)
+- 💻 **NeMo Framework:** [github.com/NVIDIA-NeMo/NeMo](https://github.com/NVIDIA-NeMo/NeMo)
+
+## Run locally
+
+```bash
+pip install -r requirements.txt
+python app.py
+```
+
+Requires an `HF_TOKEN` environment variable with access to [nvidia/magpie_tts_multilingual_357m](https://huggingface.co/nvidia/magpie_tts_multilingual_357m).
+
+Use of this model is governed by the [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license).

app.py

@@ -2,56 +2,79 @@
 To run locally, do python app.py
 """
 import os
+os.environ["NUMBA_DISABLE_CUDA"] = "1"
 os.environ['OMP_NUM_THREADS'] = '2'
 import gradio as gr
-import numpy as np
 import spaces
-from huggingface_hub import hf_hub_download
-from huggingface_hub import login
+from huggingface_hub import hf_hub_download, login
 from nemo.collections.tts.modules.magpietts_inference.utils import ModelLoadConfig, load_magpie_model
 
 login(token=os.environ.get("HF_TOKEN"))
-print(f"login successful")
+print("login successful")
+
 server_port = os.getenv("SERVER_PORT")
-print(f"server_port {server_port}")
 if isinstance(server_port, str):
     server_port = int(server_port)
+print(f"server_port {server_port}")
 
 CODEC_MODEL_PATH = "nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps"
 
-# Download a specific file
 CHECKPOINT_PATH = hf_hub_download(
     repo_id="nvidia/magpie_tts_multilingual_357m",
     filename="magpie_tts_multilingual_357m.nemo",
-    token=os.environ.get("HF_TOKEN")
+    token=os.environ.get("HF_TOKEN"),
 )
-
-print(CHECKPOINT_PATH)  # local path to the downloaded file
-
+print(f"checkpoint downloaded to {CHECKPOINT_PATH}")
+
+SPEAKER_MAP = {
+    "Aria": 0,
+    "Jason": 1,
+    "John": 2,
+    "Leo": 3,
+    "Sofia": 4,
+}
+
+LANGUAGE_MAP = {
+    "Arabic-UAE": "ar-AE",
+    "Arabic-Saudi Arabia": "ar-SA",
+    "Arabic-MSA": "ar-MSA",
+    "Chinese": "zh",
+    "English": "en",
+    "French": "fr",
+    "German": "de",
+    "Hindi": "hi",
+    "Italian": "it",
+    "Japanese": "ja",
+    "Korean": "ko",
+    "Brazilian Portuguese": "pt-BR",
+    "Spanish": "es",
+    "Vietnamese": "vi",
+}
 
 BODY = """
-**MagpieTTS** is NVIDIA's state-of-the-art multilingual text-to-speech system built on the [NeMo](https://github.com/NVIDIA/NeMo) framework. \
+**MagpieTTS** is NVIDIA's state-of-the-art multilingual text-to-speech system built on the [NeMo Speech](https://github.com/NVIDIA-NeMo/Speech) framework. \
 It employs a two-stage pipeline architecture: a language model generates discrete acoustic tokens from text, which are then decoded into \
 high-fidelity audio using a neural audio codec: [NanoCodec](https://huggingface.co/nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps). \
-The model is a text-to-speech model that generates speech in 5 different English speakers - Sofia, Aria, Jason, Leo, \
-[John Van Stan](https://librivox.org/reader/9017?primary_key=9017&search_category=reader&search_page=1&search_form=get_results&search_order=alpha). \
-Each speakers can speak nine different languages (En, Es, De, Fr, Vi, It, Zh, Hi, Ja). The model predicts discrete audio codec tokens autoregressively using \
-a transformer encoder-decoder architecture. It employs multi-codebook prediction (typically 8 codebooks) with optional local transformer refinement \
-for high-quality audio generation, and leverages techniques like attention priors, classifier-free guidance (CFG), and Group Relative Policy \
+The model synthesizes speech in 5 different English speakers - Sofia, Aria, Jason, Leo, \
+[John Van Stan](https://librivox.org/reader/9017?primary_key=9017&search_category=reader&search_page=1&search_form=get_results&search_order=alpha) \
+across 12 languages: Arabic, Chinese, English, French, German, Hindi, Italian, Japanese, Korean, Portuguese, Spanish, and Vietnamese. \
+The model predicts discrete audio codec tokens autoregressively using a transformer encoder-decoder architecture. It employs multi-codebook \
+prediction (typically 8 codebooks) with frame stacking (factor = 2) and a local transformer for fine-grained refinement of \
+high-fidelity audio generation, and leverages techniques like attention priors, classifier-free guidance (CFG), and Group Relative Policy \
 Optimization (GRPO) for improved alignment.
 
 ### Key Features of the model
-- **Multilingual Support** — Synthesizes natural speech in English, Spanish, German, French, Vietnamese, Italian, Mandarin, Hindi and Japanese.
-- **Expressive Voices** — Multiple voice options with emotional tones and gender variations including 4 proprietary voices and 1 public voice
+- **Multilingual Support** — Synthesizes natural speech in 12 languages: Arabic, Chinese, English, French, German, Hindi, Italian, Japanese, Korean, Portuguese, Spanish, and Vietnamese.
+- **Expressive Voices** — Multiple voice options with emotional tones and gender variations including 4 proprietary voices and 1 public voice.
 - **Text Normalization** — Built-in text normalization for handling numbers, abbreviations, and special characters for all languages.
 ### Resources
 - 🤗 **MagpieTTS Weights**: [nvidia/magpie_tts_multilingual_357m](https://huggingface.co/nvidia/magpie_tts_multilingual_357m)
 - 🤗 **NanoCodec Weights**: [nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps](https://huggingface.co/nvidia/nemo-nano-codec-22khz-1.89kbps-21.5fps)
-- 💻 **NeMo Framework**: [github.com/NVIDIA/NeMo](https://github.com/NVIDIA/NeMo)
+- 💻 **NeMo Framework**: [github.com/NVIDIA-NeMo/NeMo](https://github.com/NVIDIA-NeMo/NeMo)
 
 Note about the demo:
 - Text normalization takes time to load, so if you want faster generation select "Do not apply TN."
-- Text normalization works for En, Es, De, Fr, It, Zh, Vi, Hi, Ja. Added support for Vi, Hi, Ja text normalization in V26.02 version of the checkpoint.
+- Text normalization works for all supported languages.
 - Text normalization is required for numbers to be processed and spoken.
 - When the model runs on ZeroGPU Hardware, expect slower generation because the model checkpoint is loaded everytime.
 - As the model's speakers are all native English speakers, expect accented speech in the other languages.
@@ -76,83 +99,81 @@ try:
     model.eval().cuda()
     print("Model loaded and ready for inference.")
 except Exception as e:
-    print(f"exception in model loading is {e}")
-    gr.Warning(f"exception in model loading is {e}")
+    print(f"Exception in model loading: {e}")
+
 
 @spaces.GPU
 def demo_tts(input_text, language, speaker, apply_TN):
     global model
 
-    speaker_map = {
-        "John": 0,
-        "Sofia": 1,
-        "Aria": 2,
-        "Jason": 3,
-        "Leo": 4
-    }
-
-    # Validate input
     if not input_text or input_text.strip() == "":
-        gr.Warning("Please enter some text to synthesize.")
         raise gr.Error("Please enter some text to synthesize.")
 
-    # Load model on first call (inside GPU context)
+    # Reload model inside GPU context if it failed at startup (ZeroGPU fallback)
     if model is None:
-        # model = MagpieTTSModel.from_pretrained("nvidia/magpie_tts_multilingual_357m")
-        model_config = ModelLoadConfig(
-            nemo_file=CHECKPOINT_PATH,
-            codecmodel_path=CODEC_MODEL_PATH,
-            legacy_codebooks=False,
-            legacy_text_conditioning=False,
-            hparams_from_wandb=None,
-        )
         try:
             model, _ = load_magpie_model(model_config)
+            model.eval().cuda()
+            print("Model loaded and ready for inference.")
         except Exception as e:
-            gr.Warning(f"exception in model loading is {e}")
-        model.eval().cuda()
-        print("Model loaded and ready for inference.")
+            raise gr.Error(f"Model failed to load: {e}")
+
+    if speaker not in SPEAKER_MAP:
+        raise gr.Error(f"Unknown speaker: {speaker!r}. Choose from {list(SPEAKER_MAP)}.")
+    if language not in LANGUAGE_MAP:
+        raise gr.Error(f"Unknown language: {language!r}. Choose from {list(LANGUAGE_MAP)}.")
 
     apply_tn_flag = apply_TN == "Apply TN"
-    speaker_idx = speaker_map[speaker]
+    speaker_idx = SPEAKER_MAP[speaker]
+    language_code = LANGUAGE_MAP[language]
     input_text = input_text.strip()
 
     if not (input_text.endswith(".") or input_text.endswith("?") or input_text.endswith("!")):
         input_text = f"{input_text}."
+
     try:
-        audio, audio_len = model.do_tts(input_text, language=language, apply_TN=apply_tn_flag, speaker_index=speaker_idx)
+        import traceback
+        audio, audio_len = model.do_tts(
+            input_text, language=language_code, apply_TN=apply_tn_flag, speaker_index=speaker_idx
+        )
         print(f"audio {audio.shape} audio_len {audio_len}")
     except Exception as e:
-        import traceback
         traceback.print_exc()
-        gr.Error(f"exception is {e}")
+        raise gr.Error(f"Speech synthesis failed: {e}")
+
     audio_np = audio[0, :audio_len[0]].cpu().numpy()
 
     # Return only picklable objects (int and numpy array)
     return model.sample_rate, audio_np
 
-# def main():
+
 demo = gr.Interface(
     fn=demo_tts,
-    inputs=[gr.Textbox(label="Text to synthesize"),
-            gr.Dropdown(
-                choices=["en", "de", "es", "fr", "it", "vi", "zh", "hi", "ja"],
-                label="Target Language",
-                info="Select the target language for the speech to be synthesized in",
-                ),
-            gr.Dropdown(
-                choices=["Sofia", "Aria", "Jason", "Leo", "John"],
-                label="Target Speaker",
-                info="Select the target speaker whose voice you would like the the speech to be synthesized in",
-                ),
-            gr.Radio(
-                ["Apply TN", "Do not apply TN"],
-                value="Do not apply TN",
-                label="Apply Text Normalization",
-                info="Select if you want to apply text normalization to the input text"
-                ),
-            ],
+    inputs=[
+        gr.Textbox(label="Text to synthesize"),
+        gr.Dropdown(
+            choices=LANGUAGE_MAP.keys(),
+            value="English",
+            label="Target Language",
+            info="Select the target language for the speech to be synthesized in",
+        ),
+        gr.Dropdown(
+            choices=["Aria", "Jason", "John", "Leo", "Sofia"],
+            value="Aria",
+            label="Target Speaker",
+            info="Select the target speaker whose voice you would like the the speech to be synthesized in",
+        ),
+        gr.Radio(
+            ["Apply TN", "Do not apply TN"],
+            value="Do not apply TN",
+            label="Apply Text Normalization",
+            info="Select if you want to apply text normalization to the input text",
+        ),
+    ],
     outputs="audio",
     description=BODY,
-    title="Text to Speech MagpieTTS Multilingual Demo")
-demo.launch(server_port=server_port)
+    title="Text to Speech MagpieTTS Multilingual Demo",
+)
+
+if __name__ == "__main__":
+    demo.launch(server_port=server_port)

requirements.txt

@@ -1,5 +1,4 @@
-nemo_toolkit[tts]@git+https://github.com/NVIDIA/NeMo.git@f767ae5ef6a12bd0ab9173e3e536a3c6f1e5c054
-git+https://github.com/NVIDIA/NeMo-text-processing.git@0153962265c77dbd43eab5584450954d9a4f8af0
+nemo_toolkit[tts] @ git+https://github.com/blisc/NeMo.git@3685fb15405304671ffa86d8d01d2b0c9ff09bda
+git+https://github.com/NVIDIA/NeMo-text-processing.git@1f1263579fe57ba7ed783cad3dddee710fcc5064
 spaces
-gradio
 kaldialign