arXiv:2506.21298cs.SDcs.AI2025-06被引 1

对比不同适配器设计,找出低资源音乐生成的最优方案

Exploring Adapter Design Tradeoffs for Low Resource Music Generation

  • 用卷积与变压器结构适配器分别捕捉音乐细节和长程依赖
  • 中等规模适配器(4000万参数)在表达力与质量间达到最佳平衡
  • 扩散模型生成更丰富但不稳,自回归模型更快更高效

微调大规模音乐生成模型如MusicGen和Mustango成本高昂,常需更新数十亿参数。参数高效微调(PEFT)技术,特别是基于适配器的方法,成为替代方案,仅用少量可训练参数即可保持性能。本文研究了两种模型在印度古典与土耳其马卡姆音乐上的多种适配器配置,发现:基于卷积的适配器擅长捕捉装饰音、短旋律等局部细节;基于变压器的适配器更利于维持结构化即兴演奏所需的长程依赖。分析显示,4000万参数的中等规模适配器在表达力与生成质量间取得最优平衡。此外,基于扩散的Mustango生成多样性更高且更贴合输入提示,但存在音符、节奏对齐与美学稳定性不足的问题,且训练耗时显著更长。相比之下,自回归模型MusicGen训练更快、效率更高,生成质量更优,但略显冗余。

原文摘要 · Abstract (English)

Fine-tuning large-scale music generation models, such as MusicGen and Mustango, is a computationally expensive process, often requiring updates to billions of parameters and, therefore, significant hardware resources. Parameter-Efficient Fine-Tuning (PEFT) techniques, particularly adapter-based methods, have emerged as a promising alternative, enabling adaptation with minimal trainable parameters while preserving model performance. However, the design choices for adapters, including their architecture, placement, and size, are numerous, and it is unclear which of these combinations would produce optimal adapters and why, for a given case of low-resource music genre. In this paper, we attempt to answer this question by studying various adapter configurations for two AI music models, MusicGen and Mustango, on two genres: Hindustani Classical and Turkish Makam music. Our findings reveal distinct trade-offs: convolution-based adapters excel in capturing fine-grained local musical details such as ornamentations and short melodic phrases, while transformer-based adapters better preserve long-range dependencies crucial for structured improvisation. Additionally, we analyze computational resource requirements across different adapter scales, demonstrating how mid-sized adapters (40M parameters) achieve an optimal balance between expressivity and quality. Furthermore, we find that Mustango, a diffusion-based model, generates more diverse outputs with better adherence to the description in the input prompt while lacking in providing stability in notes, rhythm alignment, and aesthetics. Also, it is computationally intensive and requires significantly more time to train. In contrast, autoregressive models like MusicGen offer faster training and are more efficient, and can produce better quality output in comparison, but have slightly higher redundancy in their generations.

音乐生成适配器低资源

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。