用少量参数微调音乐大模型，效果更好更省资源。
Parameter-Efficient Transfer Learning for Music Foundation Models
- 只训练少量参数，结合探针与微调优点
- 在音乐标签任务上超越传统方法，关键检测接近微调效果
- 适合资源有限但需高效适配音乐大模型的研究者
近期发布的音乐基础模型有望实现通用、基本任务无关的音乐信息编码。常见适配下游任务的方法包括探针和微调，但各有局限：探针因冻结预训练权重导致性能不佳，微调则计算成本高且易过拟合。本文研究参数高效迁移学习（PETL）在音乐基础模型中的应用，融合探针与微调的优势。引入三类PETL方法：基于适配器、基于提示、基于重参数化的方法，仅训练少量参数，显著降低计算开销。实验表明，PETL在音乐自动标注任务上优于探针和微调；在调性检测和速度估计任务上，性能接近微调，但训练成本大幅降低。然而，当前基础模型在调性和速度任务上的表现仍存疑，因小型模型从头训练即可达到相似结果。代码已公开于 https://github.com/suncerock/peft-music/
原文摘要 · Abstract (English)
More music foundation models are recently being released, promising a general, mostly task independent encoding of musical information. Common ways of adapting music foundation models to downstream tasks are probing and fine-tuning. These common transfer learning approaches, however, face challenges. Probing might lead to suboptimal performance because the pre-trained weights are frozen, while fine-tuning is computationally expensive and is prone to overfitting. Our work investigates the use of parameter-efficient transfer learning (PETL) for music foundation models which integrates the advantage of probing and fine-tuning. We introduce three types of PETL methods: adapter-based methods, prompt-based methods, and reparameterization-based methods. These methods train only a small number of parameters, and therefore do not require significant computational resources. Results show that PETL methods outperform both probing and fine-tuning on music auto-tagging. On key detection and tempo estimation, they achieve similar results as fine-tuning with significantly less training cost. However, the usefulness of the current generation of foundation model on key and tempo tasks is questioned by the similar results achieved by training a small model from scratch. Code available at https://github.com/suncerock/peft-music/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。