用轻量适配器统一音乐生成与理解任务
Versatile Symbolic Music-for-Music Modeling via Function Alignment
- 用预训练语言模型+轻量适配器对齐音乐序列
- 在和弦识别、旋律生成等任务上表现更优
- 适合做音乐条件生成与跨模态理解研究
许多音乐AI模型学习音乐内容与人工标注标签之间的映射，但诸如和弦等标注可自然表达为符号乐谱本身（如音符序列）。这一发现使理解任务（如和弦识别）与条件生成任务（如和弦条件下的旋律生成）能统一于音乐对音乐的序列建模范式。本文提出一系列参数高效的方法，用于多种符号化音乐对音乐的任务。核心思路是：(1) 对参考序列与目标序列均使用预训练语言模型（LM），(2) 通过轻量级适配器连接两个LM。实验表明，该方法在和弦识别、旋律生成、鼓轨生成等多个任务中均取得优异性能。所有演示、代码及模型权重均已公开。
原文摘要 · Abstract (English)
Many music AI models learn a map between music content and human-defined labels. However, many annotations, such as chords, can be naturally expressed within the music modality itself, e.g., as sequences of symbolic notes. This observation enables both understanding tasks (e.g., chord recognition) and conditional generation tasks (e.g., chord-conditioned melody generation) to be unified under a music-for-music sequence modeling paradigm. In this work, we propose parameter-efficient solutions for a variety of symbolic music-for-music tasks. The high-level idea is that (1) we utilize a pretrained Language Model (LM) for both the reference and the target sequence and (2) we link these two LMs via a lightweight adapter. Experiments show that our method achieves superior performance among different tasks such as chord recognition, melody generation, and drum track generation. All demos, code and model weights are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。