无需参考文本即可零样本生成高保真语音，支持任意语音克隆与情感控制。
MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encoder
- 通过可学习说话人编码器从参考音频提取音色特征，实现零样本语音合成。
- 在语音克隆任务中达到最优的词错误率和说话人相似度，登顶TTS Arena榜单。
- 支持情绪调控、文本转语音、专业语音克隆等扩展应用，无需修改主模型。
我们提出MiniMax-Speech，一种基于自回归Transformer的文本到语音（TTS）模型，可生成高质量语音。其核心创新在于可学习的说话人编码器，能从参考音频中提取音色特征而无需转录文本，从而在零样本条件下生成与参考音色一致的高表现力语音，同时支持单次样本语音克隆且与参考语音高度相似。此外，通过提出的Flow-VAE进一步提升合成音频的整体质量。模型支持32种语言，在多项客观与主观评估中表现优异，尤其在语音克隆的客观指标（词错误率与说话人相似度）上达到当前最优（SOTA），并位居公开TTS Arena排行榜首位。得益于说话人编码器生成的鲁棒且解耦的表示，该模型具备强可扩展性：无需修改基础模型即可实现任意语音情感控制（通过LoRA）、直接从文本描述合成音色特征的文本转语音（T2V），以及通过额外数据微调音色特征的专业语音克隆（PVC）。更多示例请访问https://minimax-ai.github.io/tts_tech_report。
原文摘要 · Abstract (English)
We introduce MiniMax-Speech, an autoregressive Transformer-based Text-to-Speech (TTS) model that generates high-quality speech. A key innovation is our learnable speaker encoder, which extracts timbre features from a reference audio without requiring its transcription. This enables MiniMax-Speech to produce highly expressive speech with timbre consistent with the reference in a zero-shot manner, while also supporting one-shot voice cloning with exceptionally high similarity to the reference voice. In addition, the overall quality of the synthesized audio is enhanced through the proposed Flow-VAE. Our model supports 32 languages and demonstrates excellent performance across multiple objective and subjective evaluations metrics. Notably, it achieves state-of-the-art (SOTA) results on objective voice cloning metrics (Word Error Rate and Speaker Similarity) and has secured the top position on the public TTS Arena leaderboard. Another key strength of MiniMax-Speech, granted by the robust and disentangled representations from the speaker encoder, is its extensibility without modifying the base model, enabling various applications such as: arbitrary voice emotion control via LoRA; text to voice (T2V) by synthesizing timbre features directly from text description; and professional voice cloning (PVC) by fine-tuning timbre features with additional data. We encourage readers to visit https://minimax-ai.github.io/tts_tech_report for more examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。