VoXtream实现零延迟语音合成，从首个音素开始实时发声。
VoXtream: Full-Stream Text-to-Speech with Extremely Low Latency
- 采用增量式音素变换器与时间-深度双变换架构，实现端到端流式生成
- 初始延迟仅102毫秒，优于现有公开系统，9000小时语料训练仍保持高质
- 无需微调即可跨语种生成，适合实时语音交互应用
本文提出VoXtream，一种全自回归、零样本流式文本转语音系统，支持实时应用并可在第一个词发出时立即开始说话。该系统通过单调对齐策略和有限前瞻机制，直接将输入音素映射为音频标记，不引入延迟。其核心由增量音素变换器、预测语义与持续时间标记的时间变换器，以及生成声学标记的深度变换器构成。据作者所知，该系统在公开的流式TTS中实现了最低的初始延迟：在GPU上仅为102毫秒。尽管仅在9000小时规模语料上训练，其性能在多个指标上匹配或超越更大模型，在输出流与全流设置下均表现出竞争力。演示与代码已开放于https://herimor.github.io/voxtream。
原文摘要 · Abstract (English)
We present VoXtream, a fully autoregressive, zero-shot streaming text-to-speech (TTS) system for real-time use that begins speaking from the first word. VoXtream directly maps incoming phonemes to audio tokens using a monotonic alignment scheme and a limited look-ahead that does not delay onset. Built around an incremental phoneme transformer, a temporal transformer predicting semantic and duration tokens, and a depth transformer producing acoustic tokens, VoXtream achieves, to our knowledge, the lowest initial delay among publicly available streaming TTS: 102 ms on GPU. Despite being trained on a mid-scale 9k-hour corpus, it matches or surpasses larger baselines on several metrics, while delivering competitive quality in both output- and full-streaming settings. Demo and code are available at https://herimor.github.io/voxtream.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。