评测大模型在音乐感知任务上的听觉能力，发现其听音频不如看乐谱。
Evaluating Multimodal Large Language Models on Core Music Perception Tasks
- 用音频和乐谱分别测试大模型的音乐理解能力
- 音频上准确率远低于乐谱，逻辑推理提升有限
- 适合关注音乐AI真实听觉能力的研究者
多模态大语言模型声称具备'音乐理解'能力，但现有评估常混淆聆听与读谱。我们针对三种前沿模型（Gemini 2.5 Pro、Gemini 2.5 Flash、Qwen2.5-Omni）在三个核心音乐感知任务（切分节奏评分、转调检测、和弦性质识别）上进行基准测试，并分离三类变量：(i) 感知局限性（音频 vs. MIDI 输入），(ii) 示例暴露程度（零样本 vs. 少样本提示），(iii) 推理策略（独立、思维链、LogicLM）。其中，LogicLM 是将大模型与符号求解器结合以实现结构化推理的框架，已适配至音乐场景。结果表明存在明显感知差距：模型在 MIDI 上表现接近天花板，但在音频上准确率显著下降。推理与少样本提示带来的增益微弱。这在 MIDI 上可预期（性能已达饱和），但在音频上则出人意料——即使 LogicLM 在 MIDI 上几乎完美，仍对音频表现脆弱。在各模型中，Gemini Pro 整体表现最优。总体而言，当前系统在符号层面（如乐谱）推理良好，但尚不能可靠地从音频中‘聆听’。我们的方法与数据集使感知-推理边界清晰可见，为构建以音频为核心的稳健音乐系统提供可操作指导。
原文摘要 · Abstract (English)
Multimodal Large Language Models (LLMs) claim "musical understanding" via evaluations that conflate listening with score reading. We benchmark three SOTA LLMs (Gemini 2.5 Pro, Gemini 2.5 Flash, and Qwen2.5-Omni) across three core music skills: Syncopation Scoring, Transposition Detection, and Chord Quality Identification. Moreover, we separate three sources of variability: (i) perceptual limitations (audio vs. MIDI inputs), (ii) exposure to examples (zero- vs. few-shot manipulations), and (iii) reasoning strategies (Standalone, CoT, LogicLM). For the latter we adapt LogicLM, a framework combining LLMs with symbolic solvers to perform structured reasoning, to music. Results reveal a clear perceptual gap: models perform near ceiling on MIDI but show accuracy drops on audio. Reasoning and few-shot prompting offer minimal gains. This is expected for MIDI, where performance reaches saturation, but more surprising for audio, where LogicLM, despite near-perfect MIDI accuracy, remains notably brittle. Among models, Gemini Pro achieves the highest performance across most conditions. Overall, current systems reason well over symbols (MIDI) but do not yet "listen" reliably from audio. Our method and dataset make the perception-reasoning boundary explicit and offer actionable guidance for building robust, audio-first music systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。