用分层语言模型提升钢琴乐谱自动转录精度与效率
Piano Transcription by Hierarchical Language Modeling with Pretrained Roll-based Encoders
- 结合预训练音符编码器与分层解码器，融合帧级与语言模型优势
- 分步预测起始、音高、力度和结束时间，提升长序列处理能力
- 在两个基准数据集上显著优于传统方法，适合作为通用增强模块
自动音乐转录（AMT）旨在从原始音频中提取乐谱信息，通常采用基于音符图（piano-roll）的帧级系统或基于语言模型（LM）的逐音符预测系统。然而，帧级方法需人工设定阈值，而语言模型在处理长序列时表现不佳。本文提出一种混合方法：利用预训练的基于音符图的编码器，并结合语言模型解码器，同时采用分层预测策略——先预测起始与音高，再预测力度，最后预测终止时间。该策略将长序列分解为多个层级，降低计算开销。在两个基准音符图编码器上评估，本方法在起始-终止-力度F1分数上分别提升0.01和0.022，证明其可作为任意音符图编码器的性能增强插件。
原文摘要 · Abstract (English)
Automatic Music Transcription (AMT), aiming to get musical notes from raw audio, typically uses frame-level systems with piano-roll outputs or language model (LM)-based systems with note-level predictions. However, frame-level systems require manual thresholding, while the LM-based systems struggle with long sequences. In this paper, we propose a hybrid method combining pre-trained roll-based encoders with an LM decoder to leverage the strengths of both methods. Besides, our approach employs a hierarchical prediction strategy, first predicting onset and pitch, then velocity, and finally offset. The hierarchical prediction strategy reduces computational costs by breaking down long sequences into different hierarchies. Evaluated on two benchmark roll-based encoders, our method outperforms traditional piano-roll outputs 0.01 and 0.022 in onset-offset-velocity F1 score, demonstrating its potential as a performance-enhancing plug-in for arbitrary roll-based music transcription encoder.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。