arXiv:2607.18718eess.AS2026-07

测试大模型能否真听懂音频,而非靠文字猜答案。

Summary of DCASE 2026 Task 5: Audio-Dependent Question Answering

论文配图:Summary of DCASE 2026 Task 5: Audio-Dependent Question Answering
图 1 · 摘自论文原文
  • 用多阶段过滤法剔除仅靠文本就能答对的题目,确保问题必须依赖音频。
  • 平均准确率在真实测试集上下降11.91个百分点,说明模型普遍依赖文本幻觉。
  • 顶尖模型靠音频理解+思维链+投票策略,但仍有233题集体答错。

DCASE 2026 Task 5提出音频依赖问答(ADQA),检验大音频语言模型是否真正从音频中作答,而非依赖文本先验。通过静音音频探测、选项困惑度、大语言模型常识检查和人工审核构建的音频依赖过滤(ADF)流程,筛选出3000个必须依赖音频才能回答的题目,涵盖音乐、语音和环境声。首届任务吸引14支队伍提交36份方案,分为参数量≤100B和<10B两个赛道。首名由忠南大学团队的MOSS-Audio-8B-Thinking与Qwen3-Omni-30B集成模型取得,整体准确率58.33%;该团队纯MOSS配置在子赛道以57.30%领先。在30个开发集表现相近的提交中,测试集平均准确率下降11.91个百分点(中位数10.91),表明模型普遍依赖文本幻觉。主流架构包括MOSS-Audio-8B-Thinking(13次)、在AudioMCQ-StrongAC上使用LoRA微调,以及采用组相对策略优化(GRPO,5队)、组奖励解耦归一化策略优化(GDPO,2队)等偏好或强化学习目标。测试时几乎全部使用提示工程,多数采用多数投票或选项排列投票。所有系统均未能解答同一组233个测试题。

原文摘要 · Abstract (English)

DCASE~2026 Task~5 introduces Audio-Dependent Question Answering (ADQA), which tests whether large audio-language models answer from the audio rather than from textual priors. An Audio-Dependency Filtering (ADF) pipeline combines silent-audio probing, per-option perplexity, a large language model (LLM) commonsense check, and human review to remove items solvable from text alone. The 3000 items that pass form the ADQA-Bench evaluation set, spanning music, speech, and environmental audio. The inaugural edition draws 14 teams and 36 submissions across two tracks defined by total parameter count (up to 100B and under 10B). A Chung-Ang University ensemble of MOSS-Audio-8B-Thinking and Qwen3-Omni-30B reaches the top overall accuracy at \pct{58.33}, and a MOSS-only configuration from the same team leads the sub-10B track at \pct{57.30}. Across the 30 submissions with a comparable development score, evaluation accuracy falls by 11.91 percentage points (pp) on average (median 10.91\,pp) on the hidden evaluation split, which is designed to be harder than the development split. The most common building blocks are: the MOSS-Audio-8B-Thinking backbone (13 of 36 submissions), Low-Rank Adaptation (LoRA) fine-tuning on AudioMCQ-StrongAC, and preference or reinforcement-learning objectives -- Group Relative Policy Optimization (GRPO) in five teams, Group reward-Decoupled Normalization Policy Optimization (GDPO) in two. At test time, prompt engineering is near-universal, and majority or choice-permutation voting is common. Every system misses the same set of 233 evaluation items.

音频理解大模型评测问答系统幻觉检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。