arXiv:2409.09601cs.SDcs.AI2024-09综述被引 3

综述音乐大模型如何理解音乐语义与情感。

A Survey of Foundation Models for Music Understanding

  • 系统梳理音乐基础模型的语义理解能力
  • 分析其在跨模态融合上的表现与局限
  • 适合对音乐AI感兴趣的科研人员参考

音乐在日常生活中至关重要,满足情感与娱乐需求,并连接个人、社会与文化。更深入理解音乐可提升情绪、认知能力与文化联系。人工智能的快速发展为音乐分析提供了新途径,旨在模拟人类对音乐的理解并提供相关服务。传统模型侧重音频特征与简单任务,而近年来的大语言模型与基础模型凭借整合语义信息和强大推理能力,在捕捉复杂音乐特征、融合音乐与语言、融入丰富的音乐、情感与心理知识方面展现出潜力,有望从语义层面处理复杂音乐理解任务,输出更贴近人类感知的结果。本文是早期关于人工智能与音乐理解交叉领域的综述之一,系统调研、分析并测试了近期大规模音乐基础模型的音乐理解能力,讨论其局限性并提出未来方向,为该领域研究者提供参考。

原文摘要 · Abstract (English)

Music is essential in daily life, fulfilling emotional and entertainment needs, and connecting us personally, socially, and culturally. A better understanding of music can enhance our emotions, cognitive skills, and cultural connections. The rapid advancement of artificial intelligence (AI) has introduced new ways to analyze music, aiming to replicate human understanding of music and provide related services. While the traditional models focused on audio features and simple tasks, the recent development of large language models (LLMs) and foundation models (FMs), which excel in various fields by integrating semantic information and demonstrating strong reasoning abilities, could capture complex musical features and patterns, integrate music with language and incorporate rich musical, emotional and psychological knowledge. Therefore, they have the potential in handling complex music understanding tasks from a semantic perspective, producing outputs closer to human perception. This work, to our best knowledge, is one of the early reviews of the intersection of AI techniques and music understanding. We investigated, analyzed, and tested recent large-scale music foundation models in respect of their music comprehension abilities. We also discussed their limitations and proposed possible future directions, offering insights for researchers in this field.

音乐AI基础模型语义理解综述

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。