用可学习时钟机制实现序列对齐,提升语音合成稳定性。
Stochastic Clock Attention for Aligning Continuous and Ordered Sequences
- 引入可学习的非负时钟,将注意力建模为时钟相遇概率。
- 无需位置编码即可实现平滑、因果、近对角线的对齐效果。
- 适用于并行与自回归解码,适合语音、视频等连续序列建模。
我们提出一种针对连续有序序列的注意力机制,明确作为对齐模型,是众多序列到序列任务的核心。标准缩放点积注意力依赖位置编码和掩码,但不强制连续性或单调性,而这对于帧同步目标至关重要。我们为源和目标引入可学习的非负“时钟”,并将注意力建模为这些时钟的相遇概率;通过路径积分推导得到闭式高斯类评分规则,内在倾向于因果、平滑、近对角线的对齐,无需外部位置正则化。该框架支持两种互补模式:当全局长度已知时使用归一化时钟进行并行解码,否则使用非归一化时钟进行自回归解码——两者均几乎无额外参数,可直接替换。在Transformer文本到语音测试中,该方法生成更稳定的对齐,且对全局时间缩放更具鲁棒性,同时保持或优于缩放点积基线的准确率。我们推测其适用于其他连续目标,包括视频与时间信号建模。
原文摘要 · Abstract (English)
We formulate an attention mechanism for continuous and ordered sequences that explicitly functions as an alignment model, which serves as the core of many sequence-to-sequence tasks. Standard scaled dot-product attention relies on positional encodings and masks but does not enforce continuity or monotonicity, which are crucial for frame-synchronous targets. We propose learned nonnegative \emph{clocks} to source and target and model attention as the meeting probability of these clocks; a path-integral derivation yields a closed-form, Gaussian-like scoring rule with an intrinsic bias toward causal, smooth, near-diagonal alignments, without external positional regularizers. The framework supports two complementary regimes: normalized clocks for parallel decoding when a global length is available, and unnormalized clocks for autoregressive decoding -- both nearly-parameter-free, drop-in replacements. In a Transformer text-to-speech testbed, this construction produces more stable alignments and improved robustness to global time-scaling while matching or improving accuracy over scaled dot-product baselines. We hypothesize applicability to other continuous targets, including video and temporal signal modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。