用检索增强生成提升大模型音乐问答能力
MUST-RAG: MUSical Text Question Answering with Retrieval Augmented Generation
- 构建专用音乐向量库与检索增强框架，让大模型更好理解音乐问题
- 在音乐问答任务中显著超越传统微调方法，跨域表现更优
- 适合需要精准音乐知识的智能助手、音乐研究者使用
大型语言模型（LLMs）在多个领域展现出强大能力，但在音乐相关应用中仍受限于训练数据中音乐知识比例较低。为此，我们提出MusT-RAG框架，基于检索增强生成（RAG）技术，将通用大模型适配于纯文本音乐问答（MQA）任务。该框架通过：(1) 构建专用于音乐领域的向量数据库MusWikiDB，优化检索阶段；(2) 在推理和微调中融合上下文信息，有效实现模型的音乐专业化。实验表明，MusT-RAG在音域内和音域外音乐问答基准测试中均显著优于传统微调方法，且其构建的MusWikiDB相较于通用维基百科语料库，在性能和计算效率上均有明显提升。
原文摘要 · Abstract (English)
Recent advancements in Large language models (LLMs) have demonstrated remarkable capabilities across diverse domains. While they exhibit strong zero-shot performance on various tasks, LLMs' effectiveness in music-related applications remains limited due to the relatively small proportion of music-specific knowledge in their training data. To address this limitation, we propose MusT-RAG, a comprehensive framework based on Retrieval Augmented Generation (RAG) to adapt general-purpose LLMs for text-only music question answering (MQA) tasks. RAG is a technique that provides external knowledge to LLMs by retrieving relevant context information when generating answers to questions. To optimize RAG for the music domain, we (1) propose MusWikiDB, a music-specialized vector database for the retrieval stage, and (2) utilizes context information during both inference and fine-tuning processes to effectively transform general-purpose LLMs into music-specific models. Our experiment demonstrates that MusT-RAG significantly outperforms traditional fine-tuning approaches in enhancing LLMs' music domain adaptation capabilities, showing consistent improvements across both in-domain and out-of-domain MQA benchmarks. Additionally, our MusWikiDB proves substantially more effective than general Wikipedia corpora, delivering superior performance and computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。