arXiv:2509.09318cs.SDcs.MM2025-09中稿 · APSIPA 2025

用稀疏注意力提升钢琴转录效率,兼顾长时依赖与计算性能。

Efficient Transformer-Based Piano Transcription With Sparse Attention Mechanisms

  • 采用滑动窗口自注意力与混合全局局部交叉注意力,降低计算复杂度。
  • 在MAESTRO数据集上推理速度提升,内存占用减少,性能接近全注意力模型。
  • 适合需要高效推理的实时钢琴转录系统或资源受限场景使用。

本文研究基于计算高效且高性能的Transformer变体实现自动钢琴转录,以捕捉整首乐曲的长期依赖关系。尽管基于Transformer的序列到序列模型在钢琴转录中表现优异,但其自注意力机制具有二次复杂度,难以一次性处理整首乐曲,实际中通常采用滑动窗口方式处理音乐信号。为克服此限制,我们提出一种结合稀疏注意力机制的高效架构:编码器和解码器均引入滑动窗口自注意力,同时设计混合全局-局部交叉注意力机制,根据MIDI标记类型动态关注不同跨度;此外,在编码器与解码器间采用分层池化策略进一步降低计算负担。在MAESTRO数据集上的实验表明,该模型显著降低了计算成本与内存占用,加速了推理速度,同时保持与全注意力基线相当的转录性能。这使得在相同硬件条件下可训练更长音频上下文,验证了稀疏注意力在构建高效高精度钢琴转录系统中的可行性。代码已公开于 https://github.com/WX-Wei/efficient-seq2seq-piano-trans。

原文摘要 · Abstract (English)

This paper investigates automatic piano transcription based on computationally-efficient yet high-performant variants of the Transformer that can capture longer-term dependency over the whole musical piece. Recently, transformer-based sequence-to-sequence models have demonstrated excellent performance in piano transcription. These models, however, fail to deal with the whole piece at once due to the quadratic complexity of the self-attention mechanism, and music signals are thus typically processed in a sliding-window manner in practice. To overcome this limitation, we propose an efficient architecture with sparse attention mechanisms. Specifically, we introduce sliding-window self-attention mechanisms for both the encoder and decoder, and a hybrid global-local cross-attention mechanism that attends to various spans according to the MIDI token types. We also use a hierarchical pooling strategy between the encoder and decoder to further reduce computational load. Our experiments on the MAESTRO dataset showed that the proposed model achieved a significant reduction in computational cost and memory usage, accelerating inference speed, while maintaining transcription performance comparable to the full-attention baseline. This allows for training with longer audio contexts on the same hardware, demonstrating the viability of sparse attention for building efficient and high-performance piano transcription systems. The code is available at https://github.com/WX-Wei/efficient-seq2seq-piano-trans.

钢琴转录稀疏注意力Transformer高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。