改变自回归语音合成的生成顺序，能显著提升音质。
Decoding Order Matters in Autoregressive Speech Synthesis
- 用掩码扩散框架实验不同解码顺序，可灵活调整生成方向。
- 固定顺序（如从左到右）不如自适应顺序，音质更优。
- 1比特量化仍可生成高质量语音，适合低资源部署。
自回归语音合成通常采用从左到右的生成顺序，但这一顺序是可选的建模策略。本文通过掩码扩散框架研究解码顺序的影响，该框架可逐步解封位置，支持训练和推理阶段任意解码顺序。通过在身份排列与随机排列之间插值，发现解码顺序的随机性会影响语音质量。进一步对比固定策略（如l2r、r2l）与自适应策略（如Top-K），结果表明固定顺序（包括主流的从左到右）表现次优，而自适应解码效果更佳。此外，由于掩码扩散需离散输入，本文对声学表示进行量化，发现即使1比特量化也能支持高质量语音生成。
原文摘要 · Abstract (English)
Autoregressive speech synthesis often adopts a left-to-right order, yet generation order is a modelling choice. We investigate decoding order through masked diffusion framework, which progressively unmasks positions and allows arbitrary decoding orders during training and inference. By interpolating between identity and random permutations, we show that randomness in decoding order affects speech quality. We further compare fixed strategies, such as \texttt{l2r} and \texttt{r2l} with adaptive ones, such as Top-$K$, finding that fixed-order decoding, including the dominating left-to-right approach, is suboptimal, while adaptive decoding yields better performance. Finally, since masked diffusion requires discrete inputs, we quantise acoustic representations and find that even 1-bit quantisation can support reasonably high-quality speech.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。