229M小模型实现大模型级音乐理解能力,适合边缘设备部署。
TinyMU: A Compact Audio-Language Model for Music Understanding

- 用350万条音乐问答数据训练,结合自监督音频编码器对齐音视频特征。
- 在MuChoMusic上达到大模型82%性能,参数量仅为后者的1/35。
- 轻量化设计适合手机、耳机等资源受限的智能设备使用。
音乐理解与推理是音乐信息检索领域的重要挑战,应用涵盖检索、推荐、音乐代理和虚拟助手等。近年来的大规模音频-语言模型(LALMs)虽在回答音乐相关问题上表现优异,但其参数量常达数十亿,导致训练成本高、推理慢,难以在边缘设备部署。本文提出TinyMU,一个仅229M参数的轻量级音乐-语言模型(MLM),在保持高效紧凑的同时,性能可媲美更大模型。为训练TinyMU,我们构建了MusicSkills-3.5M数据集,包含350万条精心筛选的音乐导向问答样本,涵盖多选、二分类和开放问答三种形式,覆盖多样音乐概念。架构上,采用当前最优的自监督音频编码器MATPAC++提取细粒度特征,并搭配轻量线性投影器,高效对齐音频与语言表示。大量评估表明,TinyMU在基础音乐理解与复杂推理任务中均表现良好。尤其在MuChoMusic基准上,其性能达到当前最优大模型的82%,而模型大小仅为后者的1/35,凸显小模型在计算资源受限场景下的巨大潜力。
原文摘要 · Abstract (English)
Music understanding and reasoning are central challenges in the Music Information Research field, with applications ranging from retrieval and recommendation to music agents and virtual assistants. Recent Large Audio-Language Models (LALMs) have shown remarkable progress in answering music-related questions by following user instructions. However, their massive scale, often billions of parameters, results in expensive training, slow inference, and limited deployability on edge devices. In this work, we present TinyMU, a lightweight (229M) Music-Language Model (MLM) that achieves performance comparable to much larger LALMs while remaining efficient and compact. To train TinyMU, we introduce MusicSkills-3.5M, a carefully curated, music-grounded question-answering dataset with 3.5M samples. Spanning multiple-choice, binary, and open-ended formats, this dataset provides fine-grained supervision across diverse musical concepts. For its architecture, TinyMU leverages MATPAC++, the SOTA self-supervised audio encoder for fine-grained feature extraction. Paired with a lightweight linear projector, it efficiently aligns audio embeddings with the language model. Through extensive evaluation, we show that TinyMU performs strongly in both basic music understanding and complex reasoning. Notably, on the MuChoMusic benchmark, it achieves 82\% of SOTA LALM's performance despite being 35x smaller, highlighting the potential of small MLMs under constrained computational budgets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。