基于绝对与相对音乐属性的MIDI大模型，提升音乐生成与理解性能
Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes
- 创新分词方法+无参数相对注意力，融合音乐领域先验知识
- 在4个数据集3类任务上准确率与F1值均超越现有大模型
- 支持音乐补全等条件生成，代码模型已开源，适合音乐AI研究者
Moonbeam 是一个基于 Transformer 的符号音乐基础模型，在总计 81.6 千小时音乐和 180 亿个标记的 MIDI 数据上预训练。该模型通过新颖的领域知识启发式分词方法和多维相对注意力（MRA）机制，同时捕捉音乐的绝对与相对属性，且 MRA 不引入额外可训练参数。基于预训练的 Moonbeam，我们提出了两种具备完整前瞻能力的微调架构，分别针对符号音乐理解与条件音乐生成（包括音乐补全）两类下游任务。在 4 个数据集上的 3 个下游音乐分类任务中，模型在准确率与 F1 值方面大多优于其他大规模预训练音乐模型。此外，其微调后的条件生成模型也优于采用 REMI 式分词器的强基线 Transformer 模型。代码、预训练模型及生成样本已在 GitHub 开源。
原文摘要 · Abstract (English)
Moonbeam is a transformer-based foundation model for symbolic music, pretrained on a large and diverse collection of MIDI data totaling 81.6K hours of music and 18 billion tokens. Moonbeam incorporates music-domain inductive biases by capturing both absolute and relative musical attributes through the introduction of a novel domain-knowledge-inspired tokenization method and Multidimensional Relative Attention (MRA), which captures relative music information without additional trainable parameters. Leveraging the pretrained Moonbeam, we propose 2 finetuning architectures with full anticipatory capabilities, targeting 2 categories of downstream tasks: symbolic music understanding and conditional music generation (including music infilling). Our model outperforms other large-scale pretrained music models in most cases in terms of accuracy and F1 score across 3 downstream music classification tasks on 4 datasets. Moreover, our finetuned conditional music generation model outperforms a strong transformer baseline with a REMI-like tokenizer. We open-source the code, pretrained model, and generated samples on Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。