构建首个音乐混音协作对话数据集,支持AI理解真实混音对话。
MixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music Mixing
- 收集12名制作者在7次深度会话中的431轮音频对话
- 用该数据集微调模型后,可生成有帮助且上下文相关的混音建议
- 适合开发能辅助创作的智能混音助手,尤其助新手成长
尽管人工智能在音乐混音与母带处理中具有巨大潜力,但当前研究多聚焦于端到端自动化或生成,忽视了协同创作中至关重要的互动与教学维度,导致艺术家(尤其是希望提升技能的初学者)需求未被满足。为此,我们提出 MixAssist,一个新颖的音频-语言数据集,记录了专家与初学者在协作混音过程中产生的情境化、多轮对话。该数据集包含来自7次深度会话、12位制作者的431个音频相关对话回合,为训练和评估能够理解真实音乐制作对话复杂性的音频-语言模型提供了独特资源。我们的评估(包括自动化LLM作为裁判与人工专家对比)表明,将 Qwen-Audio 等模型在 MixAssist 上微调后可取得良好效果,其中 Qwen 显著优于其他测试模型,能生成更符合语境、更具帮助性的混音建议。通过聚焦基于音频上下文的协同教学,MixAssist 为开发旨在支持与增强音乐混音创作过程的智能AI助手提供了基础。
原文摘要 · Abstract (English)
While AI presents significant potential for enhancing music mixing and mastering workflows, current research predominantly emphasizes end-to-end automation or generation, often overlooking the collaborative and instructional dimensions vital for co-creative processes. This gap leaves artists, particularly amateurs seeking to develop expertise, underserved. To bridge this, we introduce MixAssist, a novel audio-language dataset capturing the situated, multi-turn dialogue between expert and amateur music producers during collaborative mixing sessions. Comprising 431 audio-grounded conversational turns derived from 7 in-depth sessions involving 12 producers, MixAssist provides a unique resource for training and evaluating audio-language models that can comprehend and respond to the complexities of real-world music production dialogues. Our evaluations, including automated LLM-as-a-judge assessments and human expert comparisons, demonstrate that fine-tuning models such as Qwen-Audio on MixAssist can yield promising results, with Qwen significantly outperforming other tested models in generating helpful, contextually relevant mixing advice. By focusing on co-creative instruction grounded in audio context, MixAssist enables the development of intelligent AI assistants designed to support and augment the creative process in music mixing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。