arXiv:2606.07080cs.SDcs.AI2026-06

20亿参数的连续自回归语音合成模型,支持多语言高保真生成。

dots.tts Technical Report

论文配图:dots.tts Technical Report
图 1 · 摘自论文原文
  • 用多目标AudioVAE构建语义结构化语音潜空间
  • 全历史条件流匹配降低生成漂移,提升一致性
  • 免奖励自校正训练增强鲁棒性,适合工业级部署

我们提出dots.tts,一个20亿参数的连续自回归文本到语音(TTS)基础模型,通过连续潜空间建模语音。相比现有连续自回归模型,创新有三:一、采用多目标训练的AudioVAE构建语义结构清晰且预测友好的连续语音空间;二、在流匹配头中使用全历史条件,保留长程一致性并减少生成漂移;三、对流匹配头应用免奖励自校正后训练,进一步提升鲁棒性和音质。在大规模多语言语料上训练后,dots.tts在Seed-TTS-Eval上取得最佳平均性能,中文/英文/中文难测集的字错误率(WER)分别为0.94%/1.30%/6.60%,相似度(SIM)得分分别为81.0/77.1/79.5。在其他基准上也持续表现开源最先进水平,具备强生成稳定性、语音克隆能力与情感表达力。为实现高效推理,进一步引入CFG感知的MeanFlow蒸馏,使输出流模式和双流模式下的首包延迟分别低至85毫秒和54毫秒。为促进可复现研究与实际部署,我们以Apache 2.0许可发布训练与推理代码,以及预训练、后训练和MeanFlow蒸馏的检查点。

原文摘要 · Abstract (English)

We present dots$.$tts, a 2B-parameter continuous autoregressive text-to-speech (TTS) foundation model that models speech in a continuous latent space. Compared with existing continuous autoregressive models, our key innovations are threefold. First, we train an AudioVAE with multiple objectives to build a semantically structured and prediction-friendly continuous speech space. Second, we use full-history conditioning in the flow-matching head to preserve long-range consistency and reduce drift during generation. Third, we apply reward-free self-corrective post-training to the flow-matching head to further improve robustness and acoustic quality. After being trained on a large-scale multilingual corpus, dots$.$tts achieves the best average performance on Seed-TTS-Eval, with WERs of 0.94%/1.30%/6.60% and SIM scores of 81.0/77.1/79.5 on the zh/en/zh-hard test sets, respectively. Across other benchmarks, dots$.$tts also consistently demonstrates open-source state-of-the-art performance, exhibiting strong generation stability, voice cloning ability, and emotional expressiveness. For efficient inference, we further apply CFG-aware MeanFlow distillation, enabling low-latency speech generation with first-packet latencies of 85/54 ms in output streaming and dual-streaming modes, respectively. To facilitate reproducible research and practical deployment, we release the training and inference code, together with the pretrained, post-trained, and MeanFlow-distilled checkpoints, under the Apache 2.0 license.

语音合成自回归流匹配多语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。