统一处理乐谱对齐、转录与错音检测，支持重复标记的钢琴音乐分析。
RUMAA: Repeat-Aware Unified Music Audio Analysis for Score-Performance Alignment, Transcription, and Mistake Detection
- 用三流解码器融合多任务，通过代理任务捕捉任务间依赖关系。
- 在含重复结构的乐谱上对齐效果超越现有方法，转录与错音检测表现良好。
- 适合需要高精度乐谱-演奏匹配的音乐信息检索与智能教学场景。
本文提出RUMAA，一种基于Transformer的统一音乐音频分析框架，可近端到端地完成乐谱-演奏对齐、基于乐谱的转录及错音检测。与以往分别处理这些任务的方法不同，RUMAA采用预训练的乐谱与音频编码器，并设计新颖的三流解码器，通过代理任务建模任务间的相互依赖。该方法能将可读的MusicXML乐谱（含重复符号）与完整演奏音频对齐，克服了传统基于MIDI的方法需人工展开乐谱且依赖预设重复结构的局限。在公开钢琴音乐数据集上，RUMAA在无重复乐谱上的对齐性能达到顶尖水平，在含重复乐谱上更显著优于现有方法，同时展现出有前景的转录与错音检测结果。
原文摘要 · Abstract (English)
This study introduces RUMAA, a transformer-based framework for music performance analysis that unifies score-to-performance alignment, score-informed transcription, and mistake detection in a near end-to-end manner. Unlike prior methods addressing these tasks separately, RUMAA integrates them using pre-trained score and audio encoders and a novel tri-stream decoder capturing task interdependencies through proxy tasks. It aligns human-readable MusicXML scores with repeat symbols to full-length performance audio, overcoming traditional MIDI-based methods that rely on manually unfolded score-MIDI data with pre-specified repeat structures. RUMAA matches state-of-the-art alignment methods on non-repeated scores and outperforms them on scores with repeats in a public piano music dataset, while also delivering promising transcription and mistake detection results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。