提升自回归TTS模型对长语音的稳定生成能力
Robust and Unbounded Length Generalization in Autoregressive Transformer-Based Text-to-Speech
- 引入可学习的位置对齐机制，增强跨注意力的时序感知
- 在真实语音长度下实现无重复、无遗漏的稳定输出
- 适合需要长文本高保真语音合成的应用场景
自回归Transformer模型在处理训练时未见长度的序列时表现不佳，尤其在文本转语音任务中，常出现词语重复或丢失、输出不连贯等问题。本文提出一种针对编码器-解码器结构自回归TTS系统的改进方法，通过引入可学习的相对位置对齐机制，在无需外部对齐信息的情况下，使跨注意力操作具备时序敏感性。该机制利用反向传播自动学习对齐位置，保持了交错多头自注意力与交叉注意力的灵活建模能力。所提出的Very Attentive Tacotron系统在自然度和表现力上媲美基于T5的基线模型，同时彻底解决词语重复与丢失问题，并实现任意实际长度语音的稳健泛化。
原文摘要 · Abstract (English)
Autoregressive (AR) Transformer-based sequence models are known to have difficulty generalizing to sequences longer than those seen during training. When applied to text-to-speech (TTS), these models tend to drop or repeat words or produce erratic output, especially for longer utterances. In this paper, we introduce enhancements aimed at AR Transformer-based encoder-decoder TTS systems that address these robustness and length generalization issues. Our approach uses an alignment mechanism to provide cross-attention operations with relative location information. The associated alignment position is learned as a latent property of the model via backpropagation and requires no external alignment information during training. While the approach is tailored to the monotonic nature of TTS input-output alignment, it is still able to benefit from the flexible modeling power of interleaved multi-head self- and cross-attention operations. A system incorporating these improvements, which we call Very Attentive Tacotron, matches the naturalness and expressiveness of a baseline T5-based TTS system, while eliminating problems with repeated or dropped words and enabling generalization to any practical utterance length.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。