用推测解码加速自回归语音合成，又快又准。
Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding
- 引入推测解码机制，提前预测后续语音内容。
- 推理速度显著提升，且保持高质量输出。
- 适用于多种语音数据集和编码格式，通用性强。
自回归架构（如GPT）广泛应用于现代文本转语音（TTS）系统，但其推理时间较长，尤其在处理长序列语音标记时面临挑战。本文提出VADUSA，是首个通过推测解码加速自回归TTS的方法。实验表明，VADUSA不仅大幅提高推理速度，还通过引入草稿头实现对未来语音内容的自回归预测，进一步提升性能。此外，采样过程中加入容错机制，在不损失质量的前提下进一步加速。该方法在大规模数据集及多种语音标记类型上均表现出强泛化能力。
原文摘要 · Abstract (English)
The auto-regressive architecture, like GPTs, is widely used in modern Text-to-Speech (TTS) systems. However, it incurs substantial inference time, particularly due to the challenges in the next-token prediction posed by lengthy sequences of speech tokens. In this work, we introduce VADUSA, one of the first approaches to accelerate auto-regressive TTS through speculative decoding. Our results show that VADUSA not only significantly improves inference speed but also enhances performance by incorporating draft heads to predict future speech content auto-regressively. Furthermore, the inclusion of a tolerance mechanism during sampling accelerates inference without compromising quality. Our approach demonstrates strong generalization across large datasets and various types of speech tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。