TTS-1 用大模型实现高质量实时语音合成，支持多语言情感控制。
TTS-1 Technical Report
- 基于 Transformer 的自回归模型，分两档参数量（1.6B 和 8.8B）
- 在多个基准上达最先进水平，仅靠上下文学习就还原说话人音色
- 支持 11 种语言、48kHz 高清语音与非语言发声，适合设备端部署
我们提出 Inworld TTS-1，一套基于 Transformer 的自回归文本转语音（TTS）模型。最大模型 TTS-1-Max 拥有 8.8B 参数，专为高保真和高表现力场景设计；TTS-1 模型含 1.6B 参数，面向实时语音合成与设备端应用。通过扩大训练计算量，并采用预训练、微调与强化学习对齐的序列流程优化语音语言模型（SpeechLM），两个模型在多种基准上均达到领先性能，仅依赖上下文学习即可精准复现说话人音色。TTS-1 与 TTS-1-Max 能生成 48 kHz 高分辨率语音，延迟低，支持 11 种语言，通过音频标记实现细粒度情感控制与非语言发声。相关训练与建模代码已开源，采用 MIT 许可证。
原文摘要 · Abstract (English)
We introduce Inworld TTS-1, a set of two Transformer-based autoregressive text-to-speech (TTS) models. Our largest model, TTS-1-Max, has 8.8B parameters and is designed for utmost quality and expressiveness in demanding applications. TTS-1 is our most efficient model, with 1.6B parameters, built for real-time speech synthesis and on-device use cases. By scaling train-time compute and applying a sequential process of pre-training, fine-tuning, and RL-alignment of the speech-language model (SpeechLM) component, both models achieve state-of-the-art performance on a variety of benchmarks, demonstrating exceptional quality relying purely on in-context learning of the speaker's voice. Inworld TTS-1 and TTS-1-Max can generate high-resolution 48 kHz speech with low latency, and support 11 languages with fine-grained emotional control and non-verbal vocalizations through audio markups. We additionally open-source our training and modeling code under an MIT license.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。