将自回归语音模型转为块扩散模型，实现低延迟零样本语音合成。
Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTS
- 通过先验校准评分与早停策略，解决离散语音令牌分布不均问题。
- 在零样本语音合成任务中达到与自回归模型相当的音质。
- 支持流式推理，首包延迟媲美自回归系统，实时因子显著更低。
我们提出 Chatterbox-Flash，一种通过微调预训练自回归语音解码器得到的零样本文本到语音模型，将其转换为块扩散解码器，在保持逐块流式输出的同时实现块内并行生成。研究发现，直接将主流块扩散解码应用于离散语音令牌会导致质量下降，因其长尾令牌分布会使并行位置选择偏向少数高频词。为在不修改架构的前提下缓解此问题，我们引入两种推理时技术：先验校准评分（减去块级边际令牌分布）和早停解码调度（基于校准置信度自适应终止迭代）。在标准零样本语音合成基准上，Chatterbox-Flash 达到与强自回归及非自回归基线相当的高保真度合成效果，同时支持流式推理，首包时间与流式自回归系统相当，实时因子显著降低。代码与音频样本见 https://github.com/resemble-ai/chatterbox-flash。
原文摘要 · Abstract (English)
We present Chatterbox-Flash, a zero-shot text-to-speech model obtained by fine-tuning a pretrained autoregressive TTS decoder into a block-diffusion decoder, enabling parallel token generation within each block while retaining block-by-block streaming. We find that naively transferring mainstream block-diffusion decoding to discrete speech tokens degrades quality, as a long-tail token distribution biases parallel position selection toward a few high-frequency tokens. To mitigate this without architectural modification, we introduce two inference-time techniques: prior-calibrated scoring, which subtracts the block-level marginal token distribution, and an early-decoding schedule, which adaptively terminates iteration based on calibrated confidence. On standard zero-shot TTS benchmarks, Chatterbox-Flash attains high-fidelity synthesis comparable to strong autoregressive and non-autoregressive baselines, while supporting streaming inference with time-to-first-packet on par with streaming AR systems and substantially lower real-time factor. Code and audio samples are available at https://github.com/resemble-ai/chatterbox-flash.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。