arXiv:2410.00210cs.SDcs.LG2024-10被引 16

用Transformer直接从演奏MIDI生成精准乐谱,提升记谱准确性和细节还原。

End-to-end Piano Performance-MIDI to Score Conversion with Transformers

  • 采用序列到序列的转换框架,避免逐音符分类,降低对齐与标注成本。
  • 相比之前方法,序列长度减少3.5倍,记谱准确率在MUSTER指标上显著提升。
  • 首次直接预测装饰音、符干方向等记谱细节,适合音乐信息学与智能作曲研究者。

从富有表现力的人类钢琴演奏中自动生成精确乐谱是计算音乐学的核心任务。本文提出一种端到端深度学习方法,直接从真实钢琴演奏MIDI文件构建详细乐谱。我们设计了一种基于复合标记的新型符号音乐数据分词机制,结合现代Transformer架构,将任务视为序列到序列翻译而非逐音符分类,从而降低对齐要求与标注成本,并实现更简洁准确的记谱。该分词技术通过精细量化连续值,在保留更多乐谱信息的同时,使序列长度较以往方法缩短3.5倍。使用Transformer主干网络,模型显著提升了对音符时值、节奏结构及谱表分配等细节的理解能力。在采用MUSTER等转录评估指标的端到端测试中,性能超越此前深度学习方法和复杂的基于隐马尔可夫模型的最先进系统。本方法还是首个能直接从演奏数据预测颤音标记、符干方向等记谱细节的工作。代码与模型已开源:https://github.com/TimFelixBeyer/MIDI2ScoreTransformer。

原文摘要 · Abstract (English)

The automated creation of accurate musical notation from an expressive human performance is a fundamental task in computational musicology. To this end, we present an end-to-end deep learning approach that constructs detailed musical scores directly from real-world piano performance-MIDI files. We introduce a modern transformer-based architecture with a novel tokenized representation for symbolic music data. Framing the task as sequence-to-sequence translation rather than note-wise classification reduces alignment requirements and annotation costs, while allowing the prediction of more concise and accurate notation. To serialize symbolic music data, we design a custom tokenization stage based on compound tokens that carefully quantizes continuous values. This technique preserves more score information while reducing sequence lengths by $3.5\times$ compared to prior approaches. Using the transformer backbone, our method demonstrates better understanding of note values, rhythmic structure, and details such as staff assignment. When evaluated end-to-end using transcription metrics such as MUSTER, we achieve significant improvements over previous deep learning approaches and complex HMM-based state-of-the-art pipelines. Our method is also the first to directly predict notational details like trill marks or stem direction from performance data. Code and models are available at https://github.com/TimFelixBeyer/MIDI2ScoreTransformer

乐谱生成Transformer符号音乐音频转乐谱

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。