构建300万条音乐问答数据，助力大模型理解音乐结构。
MQAD: A Large-Scale Question Answering Dataset for Training Music Large Language Models
- 用MIR与LLM自动构建音乐特征与问答对
- 覆盖27万首歌，含时变和声、段落等细节信息
- 适合音乐大模型训练，提升音频描述能力
问答是人类理解音乐的自然方式，但机器要学习需依赖大规模多样化音乐数据，而这类数据公开稀缺。本文提出MQAD，基于百万歌曲数据集（MSD），涵盖节奏、和弦、调性、结构、乐器、流派等丰富音乐特征，覆盖27万首歌曲，生成近300万条多样化的问答对与描述。该数据集独特之处在于提供时变音乐信息，如和弦与段落变化，支持深入探索歌曲内在结构。构建过程结合专用音乐信息检索（MIR）模型提取高层特征，利用大语言模型（LLM）生成自然语言问答对，并采用融合LLaMA2与Whisper架构的多模态大模型及新主观评估指标进行性能验证。实验表明，基于MQAD训练的模型在音乐音频描述任务上优于传统方法。数据与代码已开源：https://github.com/oyzh888/MQAD。
原文摘要 · Abstract (English)
Question-answering (QA) is a natural approach for humans to understand a piece of music audio. However, for machines, accessing a large-scale dataset covering diverse aspects of music is crucial, yet challenging, due to the scarcity of publicly available music data of this type. This paper introduces MQAD, a music QA dataset built on the Million Song Dataset (MSD), encompassing a rich array of musical features, including beat, chord, key, structure, instrument, and genre -- across 270,000 tracks, featuring nearly 3 million diverse questions and captions. MQAD distinguishes itself by offering detailed time-varying musical information such as chords and sections, enabling exploration into the inherent structure of music within a song. To compile MQAD, our methodology leverages specialized Music Information Retrieval (MIR) models to extract higher-level musical features and Large Language Models (LLMs) to generate natural language QA pairs. Then, we leverage a multimodal LLM that integrates the LLaMA2 and Whisper architectures, along with novel subjective metrics to assess the performance of MQAD. In experiments, our model trained on MQAD demonstrates advancements over conventional music audio captioning approaches. The dataset and code are available at https://github.com/oyzh888/MQAD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。