多标记预测与推测解码加速语音合成，速度提升4-5倍且音质不降。
Accelerating Codec-based Speech Synthesis with Multi-Token Prediction and Speculative Decoding
- 单步预测多个标记，用多头结构线性提速。
- 引入维特比算法优化生成序列，减少冗余计算。
- 无需额外训练，适合追求高效语音合成的开发者。
本文旨在以最小质量损失加速基于编解码器的语音合成系统。提出一种增强型推理方法，可在推理过程中灵活权衡速度与质量，无需额外训练。核心思想是通过多个预测头在自回归模块中每步预测多个标记，使合成时间随头数增加呈线性下降。此外，引入一种新的推测解码技术，利用基于维特比算法的策略在每一步选择最优生成标记序列。实验表明，与基线模型相比，每个标记的预测时间减少4至5倍，且语音可懂度无明显下降甚至有所提升。
原文摘要 · Abstract (English)
The goal of this paper is to accelerate codec-based speech synthesis systems with minimum sacrifice to speech quality. We propose an enhanced inference method that allows for flexible trade-offs between speed and quality during inference without requiring additional training. Our core idea is to predict multiple tokens per inference step of the AR module using multiple prediction heads, resulting in a linear reduction in synthesis time as the number of heads increases. Furthermore, we introduce a novel speculative decoding technique that utilises a Viterbi-based algorithm to select the optimal sequence of generated tokens at each decoding step. In our experiments, we demonstrate that the time required to predict each token is reduced by a factor of 4 to 5 compared to baseline models, with minimal quality trade-off or even improvement in terms of speech intelligibility. Audio samples are available at: multpletokensprediction.github.io/multipletokensprediction.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。