提出新型位置编码方法,让语音增强Transformer模型更好处理更长语音。
Exploring Length Generalization For Transformer-based Speech Enhancement
- 用相对位置编码提升模型对长语音的适应能力
- 新方法LearnLin仅用一个参数每头,性能优于现有方案
- 适合需要处理超长语音的实际语音增强场景
Transformer架构在语音增强中表现优异,但其核心自注意力机制具有二次复杂度,难以训练长语音。实际应用中,运行时语音常远超训练长度,如何实现长度泛化仍是挑战。本文通过大量实验探索不同训练目标与五种评估指标下的长度泛化能力。结果表明,位置编码是缓解语音长度影响的关键。对比多种位置编码方法发现,相对位置编码优于绝对编码。进一步提出简化有效的新方案LearnLin:每个注意力头仅用一个可学习参数来缩放帧间真实相对位置,自动学习短/长依赖偏好。实验显示该方法在长度泛化上表现卓越,性能不亚于甚至超越现有最先进方法。
原文摘要 · Abstract (English)
Transformer network architecture has proven effective in speech enhancement. However, as its core module, self-attention suffers from quadratic complexity, making it infeasible for training on long speech utterances. In practical scenarios, speech enhancement models are often required to perform on noisy speech at run-time that is substantially longer than the training utterances. It remains a challenge how a Transformer-based speech enhancement model can generalize to long speech utterances. In this paper, extensive empirical studies are conducted to explore the model's length generalization ability. In particular, we conduct speech enhancement experiments on four training objectives and evaluate with five metrics. Our studies establish that positional encoding is an effective instrument to dampen the effect of utterance length on speech enhancement. We first explore several existing positional encoding methods, and the results show that relative positional encoding methods exhibit a better length generalization property than absolute positional encoding methods. Additionally, we also explore a simpler and more effective positional encoding scheme, i.e. LearnLin, that uses only one trainable parameter for each attention head to scale the real relative position between time frames, which learns the different preferences on short- or long-term dependencies of these heads. The results demonstrate that our proposal exhibits excellent length generalization ability with comparable or superior performance than other state-of-the-art positional encoding strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。