改进位置编码，让语音合成对文本长度更敏感，生成更准更快。
Length-Aware Rotary Position Embedding for Text-Speech Alignment
- 用归一化长度的相对位置替代绝对索引，增强对长短句的适应性。
- 在30秒长语音上表现稳定，错误率低于现有方法，收敛更快。
- 适合需要高精度语音合成和长语音生成的场景，如语音助手、有声书。
近年来许多文本到语音（TTS）系统基于Transformer架构，使用交叉注意力机制实现文本与语音对齐。其中，旋转位置编码（RoPE）常用于编码文本与语音表示中的位置信息。本文提出一种简单而有效的RoPE扩展——长度感知旋转位置编码（LARoPE），通过使用归一化的长度索引计算查询与键之间的相对距离，而非依赖绝对索引。实验表明，LARoPE在多个方面均优于传统RoPE：损失收敛更快，文本-语音对齐更准确，整体语音质量更高。尤其在不同语句时长变化下更具鲁棒性，最长可支持30秒语音生成，而RoPE性能显著下降。此外，本方法在标准零样本TTS基准上达到最优词错误率。
原文摘要 · Abstract (English)
Many recent text-to-speech (TTS) systems are built on transformer architectures and employ cross-attention mechanisms for text-speech alignment. Within these systems, rotary position embedding (RoPE) is commonly used to encode positional information in text and speech representations. In this work, we introduce length-aware RoPE (LARoPE), a simple yet effective extension of RoPE that improves text-speech alignment. Unlike RoPE, which relies on absolute indices, LARoPE computes relative distances between query and key positions using length-normalized indices. Experimental results show that LARoPE consistently outperforms RoPE, offering faster loss convergence, more accurate text-speech alignment, and higher overall TTS quality. Furthermore, LARoPE demonstrates greater resilience to variations in utterance duration and maintains stable performance in extended speech generation up to 30 seconds, whereas RoPE suffers from notable degradation. Notably, our method achieves a state-of-the-art word error rate on a standard zero-shot TTS benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。