用块离散扩散模型实现高效高质语音合成，支持并行生成。
DLLM-TTS: Block Discrete Diffusion Language Model for Text-to-Speech Synthesis
- 将语音合成建模为分块的离散扩散过程，兼顾局部音色一致性和全局文本对齐。
- 0.6B参数模型在2万小时数据上训练，实时因子达0.15，性能媲美主流系统。
- 适合需要快速生成且保持高可懂度的语音应用，如语音助手、内容创作。
当前语音合成系统面临权衡：自回归编解码语言模型生成语音清晰度高，但需大规模模型与数据，且逐个生成音素；非自回归方法虽加速，却牺牲语言准确性。本文提出DLLM-TTS，将文本到语音合成建模为在X-Codec2神经音频编码器的离散令牌上进行条件块离散扩散。模型将序列分块，在每块内采用掩码扩散，同时按块顺序处理，从而学习局部声学一致性与全局文本-语音对齐关系。推理时，块内并行预测音素，实现高效生成，实时因子（RTF）达0.15。一个0.6B参数模型在20,000小时数据上训练，在Seed-TTS-eval基准上表现优异，表明块离散扩散语言模型能实现高效、数据节省且支持并行生成的语音合成。
原文摘要 · Abstract (English)
Current text-to-speech systems face a trade-off: autoregres- sive codec language models produce highly intelligible speech but require large-scale models and training data and decode tokens sequentially, while non-autoregressive approaches im- prove speed at the cost of linguistic accuracy. We present DLLM-TTS, a framework that formulates TTS as conditional block discrete diffusion over X-Codec2 neural audio codec to- kens. The model decomposes sequences into blocks and applies masked diffusion within each block while processing blocks se- quentially, learning both local acoustic coherence and global text-speech alignment. During inference, parallel token pre- diction within blocks enables efficient generation with a real- time factor (RTF) of 0.15. A 0.6B-parameter model trained on 20K hours achieves competitive performance on the Seed- TTS-eval benchmark, demonstrating that block discrete diffu- sion language models enable practical and data-efficient speech synthesis with parallel generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。