将音频转为可读符号特征，提升大模型音频推理的可解释性。
SAR-LM: Symbolic Audio Reasoning with Large Language Models
- 把语音、音效、音乐转为结构化符号输入，替代密集嵌入
- 在三个基准上表现接近顶尖水平，错误可定位到具体特征
- 适合需要透明推理的场景，如医疗或自动驾驶音频分析
大型语言模型在文本和视觉任务中取得进展，但其音频推理能力仍受限。现有方法多依赖难以解释的密集音频嵌入，常在结构化推理任务中失败。基于最新基准MMAU提出的基于字幕的方法虽提升性能，但仍以密集嵌入为输入，无法提供故障分析洞察。本文提出SAR-LM，一种符号化音频推理流程，在字幕基础上将音频转化为跨语音、声音事件和音乐的结构化、人类可读特征。此类符号输入支持推理与透明错误分析，可追踪失败至具体特征。在MMAU、MMAR和OmniBench三个基准上，SAR-LM取得具有竞争力的结果，其核心贡献在于可解释性。
原文摘要 · Abstract (English)
Large language models (LLMs) have advanced in text and vision, but their reasoning on audio remains limited. Most existing methods rely on dense audio embeddings, which are difficult to interpret and often fail on structured reasoning tasks. Caption-based approaches, introduced in recent benchmarks such as MMAU, improve performance by translating audio into text, yet still depend on dense embeddings as input, offering little insight when models fail. We present SAR-LM, a symbolic audio reasoning pipeline that builds on this caption-based paradigm by converting audio into structured, human-readable features across speech, sound events, and music. These symbolic inputs support both reasoning and transparent error analysis, enabling us to trace failures to specific features. Across three benchmarks, MMAU, MMAR, and OmniBench, SAR-LM achieves competitive results, while prioritizing interpretability as its primary contribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。