用CTC对齐替代MFA，实现低延迟高质量语音合成
CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment
- 用CTC构建语音-文本对齐器，替代传统MFA工具
- 提出双词交错策略，提升流式合成效果与零样本能力
- 兼顾高音质（CTC-TTS-L）与低延迟（CTC-TTS-F），适合实时语音应用
基于大语言模型（LLM）的文本转语音（TTS）系统能生成自然语音，但多数不适用于低延迟的双流合成。高质量双流TTS依赖精确的语音-文本对齐和兼顾质量与延迟的训练序列设计。以往方法多依赖基于GMM-HMM的强制对齐工具（如MFA），流程复杂且灵活性差；固定比例交错文本与语音标记难以捕捉对齐规律。本文提出CTC-TTS，用基于CTC的对齐器替代MFA，引入基于双词的交错策略。设计两种变体：CTC-TTS-L（沿序列长度拼接标记）用于更高音质，CTC-TTS-F（沿特征维度堆叠嵌入）用于更低延迟。实验表明，CTC-TTS在流式合成和零样本任务上均优于固定比例交错和MFA基线。语音样例可访问 https://ctctts.github.io/。
原文摘要 · Abstract (English)
Large-language-model (LLM)-based text-to-speech (TTS) systems can generate natural speech, but most are not designed for low-latency dual-streaming synthesis. High-quality dual-streaming TTS depends on accurate text--speech alignment and well-designed training sequences that balance synthesis quality and latency. Prior work often relies on GMM-HMM based forced-alignment toolkits (e.g., MFA), which are pipeline-heavy and less flexible than neural aligners; fixed-ratio interleaving of text and speech tokens struggles to capture text--speech alignment regularities. We propose CTC-TTS, which replaces MFA with a CTC based aligner and introduces a bi-word based interleaving strategy. Two variants are designed: CTC-TTS-L (token concatenation along the sequence length) for higher quality and CTC-TTS-F (embedding stacking along the feature dimension) for lower latency. Experiments show that CTC-TTS outperforms fixed-ratio interleaving and MFA-based baselines on streaming synthesis and zero-shot tasks. Speech samples are available at https://ctctts.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。