arXiv:2503.01362cs.SDcs.IR2025-03中稿 · ISMIR 2024被引 2

提出一种实时钢琴转录模型,区分起音和止音检测,提升精度并降低计算开销。

Streaming Piano Transcription Based on Consistent Onset and Offset Decoding with Sustain Pedal Detection

  • 分设起音与止音双解码器,结合延音踏板状态验证,增强事件对应关系。
  • 在MAESTRO数据集上达到或超过离线最优方法的转录准确率,延迟更低。
  • 适合需要低延迟实时钢琴转录的应用,如现场演奏同步、音乐教育系统。

本文提出一种基于流式处理的音频到MIDI钢琴转录方法,旨在将音乐信号逐帧转换为音符起音和止音事件序列。传统序列到序列方法常采用计算量大的Transformer模型,虽在离线基准中表现优异,但通过因果注意力机制扩展至流式场景仍存在性能瓶颈。我们发现问题主要出在解码器:起音与止音检测依赖的时间-频率特征差异显著,而单一解码器需输出混合事件序列,无法保证同一音符起止事件的正确配对。为此,我们设计了一种流式编码器-解码器架构,使用卷积编码器聚合局部声学特征,随后由一个自回归Transformer解码器检测可变数量的起音事件,并由另一解码器针对活跃音高检测止音事件,同时每帧验证延音踏板状态。在MAESTRO数据集上的实验表明,该方法性能达到甚至优于当前最先进的离线方法,同时显著降低计算成本。

原文摘要 · Abstract (English)

This paper describes a streaming audio-to-MIDI piano transcription approach that aims to sequentially translate a music signal into a sequence of note onset and offset events. The sequence-to-sequence nature of this task may call for the computationally-intensive transformer model for better performance, which has recently been used for offline transcription benchmarks and could be extended for streaming transcription with causal attention mechanisms. We assume that the performance limitation of this naive approach lies in the decoder. Although time-frequency features useful for onset detection are considerably different from those for offset detection, the single decoder is trained to output a mixed sequence of onset and offset events without guarantee of the correspondence between the onset and offset events of the same note. To overcome this limitation, we propose a streaming encoder-decoder model that uses a convolutional encoder aggregating local acoustic features, followed by an autoregressive Transformer decoder detecting a variable number of onset events and another decoder detecting the offset events for the active pitches with validation of the sustain pedal at each time frame. Experiments using the MAESTRO dataset showed that the proposed streaming method performed comparably with or even better than the state-of-the-art offline methods while significantly reducing the computational cost.

钢琴转录流式处理音频生成多任务解码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。