用轻量模型预估语音生成候选序列，加速推理且保持音质。
Accelerating Autoregressive Speech Synthesis Inference With Speech Speculative Decoding
- 用轻量模型先生成候选语音标记序列
- 目标模型并行验证，实现1.4倍加速
- 适合对实时性有要求的语音合成场景
当前基于语言模型的自回归语音合成模型表现优异，但其逐词生成的特性导致推理延迟高，限制了在低延迟场景的应用。本文提出语音推测解码（SSD），通过轻量级草稿模型生成候选标记序列，并由目标模型在并行框架下进行验证。实验表明，SSD相比传统自回归解码实现了1.4倍的速度提升，同时保持高保真度与自然度。主观评估进一步验证了该方法在加速推理的同时有效保留了目标模型的感知质量。
原文摘要 · Abstract (English)
Modern autoregressive speech synthesis models leveraging language models have demonstrated remarkable performance. However, the sequential nature of next token prediction in these models leads to significant latency, hindering their deployment in scenarios where inference speed is critical. In this work, we propose Speech Speculative Decoding (SSD), a novel framework for autoregressive speech synthesis acceleration. Specifically, our method employs a lightweight draft model to generate candidate token sequences, which are subsequently verified in parallel by the target model using the proposed SSD framework. Experimental results demonstrate that SSD achieves a significant speedup of 1.4x compared with conventional autoregressive decoding, while maintaining high fidelity and naturalness. Subjective evaluations further validate the effectiveness of SSD in preserving the perceptual quality of the target model while accelerating inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。