groxaxo/MOSS-TTS-v1.5-GPTQ-Pro-4bit-Fused

MOSS-TTS v1.5 — GPTQPro4-Fused

Project preference: GPTQ Pro fused. The owner preferred this runtime after reviewing the audio. This is a subjective project choice, not evidence that it wins every objective score. The fused Triton range guard is part of the tested runtime; it uses the same packed weights as the normal GPTQ adapter and generated byte-identical WAVs for all 16 matched cases.

This is a native custom checkpoint, with measured RTX 3090 audio results and a successful fresh-install RTX 3060 GPU audio test. It requires the supplied runtime adapter and the pinned GPTQ-Pro backend. It is not a standard AutoGPTQ/Transformers checkpoint or a drop-in file for llama.cpp, vLLM, or an unmodified EXL3 model loader. Review the custom Python code before running it.

Findings

Runtime	Synthesis RTF	Peak allocated / reserved GiB	WavLM	ECAPA	Predicted UTMOS	CER / WER
BF16	0.558	19.50 / 19.91	0.9703	0.6210	3.126	1.13% / 3.25%
AWQ4-G64	1.212	10.66 / 11.07	0.9685	0.5807	3.177	1.63% / 4.27%
GPTQPro4	0.759	10.04 / 10.50	0.9694	0.6119	3.120	2.14% / 5.69%
GPTQPro4-Fused	0.605	10.04 / 10.48	0.9694	0.6119	3.120	2.14% / 5.69%
EXL3-4	0.518	9.79 / 9.84	0.9652	0.6135	3.127	2.18% / 4.88%
EXL3-5	0.525	10.56 / 10.62	0.9721	0.6405	3.111	1.13% / 3.66%

RTF is synthesis time divided by resulting audio duration; it includes actual co-resident FP32 decoding. Lower RTF and memory are better. WavLM/ECAPA are cosines against original reference recordings, not percentages. UTMOS is predicted naturalness, not human MOS. CER/WER measure generated-audio intelligibility through Whisper-small and include recognizer errors.

There are 16 cases per runtime: two existing female reference voices, three Spanish prompts and one English prompt, two seeds (1234 and 501). Total: 96 render records, 80 unique WAV hashes; the two GPTQ runtimes are not independent audio samples. References were 15 s and 11.008 s and were not recordings of the intended callers. All five UTMOS folds, original-reference WavLM/ECAPA with alternate-voice controls, finite audio, EOS and hashes were verified. Each native quantization checked 252 modules against its reconstructed quantized weights, minimum cosine at least 0.999; that is native-kernel parity, not BF16 equivalence.

EXL3-5 measured in a later GPU window, with other-GPU activity. Small timing differences are not controlled speed evidence. The tested AWQ path dequantizes to BF16 GEMM; it is not an optimized Marlin result. Do not generalize that timing to every AWQ implementation.

English remains a screening concern. Only four cases and one English prompt per runtime were tested. Predicted English UTMOS: BF16 3.323, GPTQ 3.137, EXL3-4 3.078, EXL3-5 3.090. English CER: BF16 0%, GPTQ/EXL3-4 4.55%, EXL3-5 1.82%. Validate both languages with the actual callers and blinded listening. Higher mean reference cosine does not establish a universal identity or quality winner.

Fresh-install RTX 3060 result

A clean Python 3.12 environment, anonymous public checkpoint download and newly compiled native GPTQ Pro extension generated all 16 matched clips successfully on a 12 GB RTX 3060. Pooled generation/synthesis RTF: 1.535 / 1.552; peak allocated/reserved VRAM: 10.040 / 10.480 GiB. The earlier 3090 synthesis RTF was 0.605: this 3060 host used 2.567 times as much timed synthesis per second of audio and did not sustain real time.

Short Spanish cases averaged 4.56 s audio, available as complete utterances after 9.12 s of timed synthesis. The host CPU was an i7-4790; CPU, GPU and measurement-window differences are part of this comparison. All 16 WAV hashes differ from the 3090 outputs, so earlier quality scores are not assigned to the new audio. Streaming, simultaneous calls and 3060 quality rescoring were not tested. Fresh-install procedure and caveats, per-clip metrics.

Format and tensor allocation

All 252 transformer linears at 4 bits; text embedding and all 33 heads affine 8-bit G64; audio embeddings/norms original BF16.

GPTQ uses native symmetric W4/G64 and FP16 kernel scales, with a per-row power-of-two input guard and FP32 restoration to BF16. This publication uses the fused Triton guard. EXL3 uses native calibrated LDLQ trellis coding with no affine group-64 or zero-point equivalent. The same backbone tensors are selected; quantized distributions and algorithms differ. Weights were converted directly from pinned BF16, never by requantizing AWQ or EXL3-4.

Weight SHA-256: e48be4962999023b7a57601b301385b0a02d5452b4ea72e4f5e48fdf6febcc37. Tensor payload: 5,516,332,016 bytes; effective 5.1981 bits per original parameter including protected tensors. Safetensors container size is slightly larger.

Only the generator checkpoint is distributed. Download the original FP32 codec separately at revision 3cd226ba2947efa357ef453bcad111b6eafba782; generator and codec both contribute to reported VRAM. The MOSS checkpoint has no compatible trained MTP path; its 33 text/RVQ heads are not future-token predictors. Matched W4A8 was not tested because the installed QQQ kernel lacks G64 support.

Download and verify

from huggingface_hub import snapshot_download
model_dir = snapshot_download("groxaxo/MOSS-TTS-v1.5-GPTQ-Pro-4bit-Fused")

Run python verify_bundle.py inside the snapshot to verify all distributed file hashes and checkpoint structure. This does not execute CUDA or rerun unavailable private-reference media.

Native runtime

The measured environment is listed in tested-environment.txt: Linux, Python 3.12, Torch 2.11.0+cu130, Transformers 5.19.0 and Triton 3.6.0 on one RTX 3090. EXL3 compilation used CUDA toolkit 12.8. Install the pinned backend using its own build instructions:

• GPTQ-Pro: groxaxo/GPTQ-Pro, import gptqmodel.
• EXL3: turboderp-org/exllamav3, import exllamav3.

Install the dependency matching this model. No native binary is bundled. The loader uses its native packed modules plus protected affine embedding/heads; ordinary AutoModel.from_pretrained is not supported for these weights.

Download the codec:

from huggingface_hub import snapshot_download
codec_dir = snapshot_download("OpenMOSS-Team/MOSS-Audio-Tokenizer", revision="3cd226ba2947efa357ef453bcad111b6eafba782")
print(codec_dir)

Then, from this model snapshot on a GPU you have reserved:

CUDA_VISIBLE_DEVICES=0 python infer.py --codec-dir /path/to/codec --reference /path/to/consented-reference.wav --language Spanish --text "Hola, me alegra hablar con vos." --output output.wav

Use --language English for English text. The CLI encodes the reference with FP32, unloads the encoder, loads the native generator and FP32 decoder, then returns the complete waveform. The public package passed a fresh-install RTX 3060 CUDA/audio smoke test on 27 October 2026, followed by 16 matched audio cases. See installation steps and limitations, including the documented NumPy dependency-metadata exception. This validates this tested installation; live-call deployment remains untested.

Practical limits

Full-utterance RTF is not streaming time to first audio, simultaneous-speaker throughput or telephone latency. Actual two-way conversations, 30-second enrolment, physical iPhone audio routing, caller-specific resemblance and human preference tests remain unvalidated. Allow cold-start headroom: EXL3-5 setup/reload reporting reached 17.19 GiB allocated / 17.96 GiB reserved, above its 10.56 GiB synthesis peak. Do not assume ASR co-residency from isolated TTS numbers.

Use reference recordings with the speaker's permission. No voice WAVs, cached reference codes, speaker embeddings, calibration tensors, evaluator weights or credentials are in this repository. Recreating bit-identical quantization requires the original permitted calibration inputs, which are not distributed; their hashes and protocol are recorded.

Evidence and attribution

Grouped voice/language metrics, per-clip metrics, validation receipt, model provenance. Machine identifiers/absolute paths were redacted from public metrics; numeric measurements are unchanged. The benchmark source archive at experimentos, commit 4f9339c may require access; the relevant metric evidence is included here.

Base model: OpenMOSS-Team/MOSS-TTS-v1.5, Apache-2.0 per the pinned model card. This is a community quantization, not an official OpenMOSS release. See LICENSE, NOTICE, UPSTREAM-MODEL-CARD.md and licenses/ for attribution.