arXiv:2608.08638cs.SDcs.AI2026-08

CuteTTS实现低延迟高保真语音合成,适合实时交互应用。

CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents

论文配图:CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
图 1 · 摘自论文原文
  • 用连续潜变量与分块自回归建模,提升语音细节与生成效率。
  • 零样本克隆语音在两个数据集上表现优秀,首次音频延迟降低23.3%。
  • 适合需要快速响应的语音助手、个性化媒体等实时场景。

零样本文本转语音(TTS)已广泛用于交互式助手、个性化媒体和无障碍工具。所有TTS系统需具备准确的语言表达、一致的说话人特征以及低延迟响应。然而,紧凑的流式系统必须在可预测的低速率潜变量序列中保留足够声学细节,而迭代扩散采样与无分类器引导会显著增加每步自回归推理成本。为平衡高保真合成与低延迟推理,我们提出CuteTTS,一种紧凑的连续自回归TTS系统。它结合语义对齐的因果变分自编码器潜变量、分块级自回归建模、显式说话人条件控制及双向流匹配头。此外,引入引导步骤蒸馏,将无分类器引导与多求解器步骤整合为单个区间条件学生模型。在LibriSpeech与Seed-TTS-Eval上的评估显示,其在零样本语音克隆中具有竞争力的可懂度与说话人相似性;蒸馏使首次音频延迟降低23.3%,实时因子下降40.8%,同时保持相当的客观与主观质量。该结果为连续自回归TTS提供了兼顾高保真生成与实时交互需求的实用路径。

原文摘要 · Abstract (English)

Zero-shot text-to-speech (TTS) now supports interactive assistants, personalized media, and accessibility tools. All TTS systems require faithful linguistic rendering, consistent speaker identity, and low-latency response. Yet compact streaming systems must preserve sufficient acoustic detail in a predictable low-rate latent sequence, while iterative diffusion sampling and classifier-free guidance multiply inference cost at every autoregressive step. To strike a balance between high-fidelity synthesis and low-latency inference, we present CuteTTS, a compact continuous-autoregressive TTS system. It combines semantically aligned causal VAE latents with patch-level autoregression, explicit speaker conditioning, and a bidirectional flow-matching head. We further introduce guidance-step distillation, which absorbs classifier-free guidance and multiple solver steps into a single interval-conditioned student. Evaluations on LibriSpeech and Seed-TTS-Eval demonstrate competitive intelligibility and speaker similarity in zero-shot voice cloning, while distillation lowers first-audio latency by 23.3% and real-time factor by 40.8% relative to the base model with comparable objective and subjective quality. These results provide a practical path toward continuous-autoregressive TTS that reconciles high-fidelity generation with the latency demands of real-time interaction.

语音合成自回归模型低延迟零样本克隆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。