实现真正逐词流式语音合成,低延迟且音质接近离线模型。
X2Streaming-TTS: Causal Token-Level Text-to-Speech from Streaming Text with Speech-State Inheritance

- 通过因果承诺机制处理不确定文本前缀,动态缓冲并自适应分段。
- 引入语音状态继承,跨段保持声学连续性,上下文有限且不越界。
- 实测单请求首音频延迟仅15.8毫秒,适合实时对话系统使用。
流式文本转语音对低延迟语音对话系统至关重要,但许多系统需等待完整句子,仅为伪流式。真正的逐词合成需在文本不完整时生成语音,同时在无限流中维持感知连贯性,且上下文受限。本文提出X2Streaming-TTS,一种因果型TTS框架,可异步接收文本令牌并即时输出语音,无需访问未来输入。为应对不确定前缀,提出因果承诺机制,通过感知不确定性的缓冲与自适应、标点感知的分段策略,暂存模糊表达。为保持声学连续性,引入因果语音状态继承,跨段传递完整的Code2Wav状态及选定的历史说话人状态。结合注意力先验约束,禁止访问未来位置的同时保留有限声学上下文。实验表明,该模型在多数主观与客观指标上优于现有伪流式模型。进一步分析显示,因果承诺稳定了在线分段,减少因上下文不足导致的失败;语音状态继承提升了边界连续性,且未降低自然度或说话人辨识度。最终,该方法实现了严格逐词流式合成,音质媲美离线基线,单请求中位首音频延迟(TTFT)为15.8毫秒,在128并发请求下中位TTFT为260.8毫秒。代码已开源:https://github.com/X-Square-Robot/X2Streaming-TTS。
原文摘要 · Abstract (English)
Streaming text-to-speech is essential for low-latency spoken dialogue systems, yet many systems wait for sentence-level text and are therefore only pseudo-streaming. True token-level synthesis must generate speech from uncertain prefixes while maintaining perceptual continuity over an unbounded stream with bounded context. We present X2Streaming-TTS, a causal TTS framework that consumes asynchronously arriving text tokens and emits speech without accessing future input. To handle uncertain prefixes, we introduce causal commitment, which keeps ambiguous expressions provisional through uncertainty-aware buffering and performs capacity-adaptive, punctuation-aware segmentation. To preserve acoustic continuity, we further introduce causal speech-state inheritance, which carries the complete Code2Wav state and selected historical Talker states across segment boundaries. Together with an attention prior constraint, it blocks access to future positions while retaining bounded acoustic context. Experiments show that X2Streaming-TTS outperforms existing pseudo-streaming models on most subjective and objective metrics. Further analysis shows that causal commitment stabilizes online segmentation and reduces failures caused by insufficient context, while speech-state inheritance improves boundary continuity without degrading naturalness or speaker identity. X2Streaming-TTS thus achieves strict token-level synthesis with quality comparable to the evaluated offline baselines, a median time to first audio token (TTFT) of 15.8 ms for a single request, and a median TTFT of 260.8 ms at 128 concurrent requests. Our implementation is publicly available at https://github.com/X-Square-Robot/X2Streaming-TTS .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。