用反事实难例训练音频模型，减少幻觉，提升听懂真实事件的能力。
AHA: Aligning Large Audio-Language Models for Reasoning Hallucinations via Counterfactual Hard Negatives
- 通过反事实难例挖掘构建高质量偏好数据集，强化模型对真实音频证据的区分能力。
- 在自建诊断集AHA-Eval上提升13.7%，并在MMAU-Test和MMAR上分别提高1.3%和1.6%。
- 方法可泛化至公开基准，适合需要高可信音频理解的多模态应用开发者使用。
尽管大型音频语言模型（LALMs）表现达到顶尖水平，但常出现幻觉，如生成与音频输入无关的内容。我们分析了此类接地失败，并提出四类具体类型：事件遗漏、错误事件身份、时间关系错误和定量时间错误。为此，我们提出AHA（Audio Hallucination Alignment）框架，利用反事实难例挖掘构建高质量偏好数据集，使模型能严格区分真实声学证据与语言上合理的虚构内容。同时，我们建立AHA-Eval诊断基准，以严格测试模型的细粒度时间推理能力。将该数据用于对Qwen2.5-Omni进行对齐，得到的Qwen-Audio-AHA模型在AHA-Eval上提升13.7%。更重要的是，这一优势可泛化至公共基准，在MMAU-Test上提升1.3%，在MMAR上提升1.6%，优于最新SOTA方法。模型与数据集已开源：https://github.com/LLM-VLM-GSL/AHA。
原文摘要 · Abstract (English)
Although Large Audio-Language Models (LALMs) deliver state-of-the-art (SOTA) performance, they frequently suffer from hallucinations, e.g. generating text not grounded in the audio input. We analyze these grounding failures and identify a distinct taxonomy: Event Omission, False Event Identity, Temporal Relation Error, and Quantitative Temporal Error. To address this, we introduce the AHA (Audio Hallucination Alignment) framework. By leveraging counterfactual hard negative mining, our pipeline constructs a high-quality preference dataset that forces models to distinguish strict acoustic evidence from linguistically plausible fabrications. Additionally, we establish AHA-Eval, a diagnostic benchmark designed to rigorously test these fine-grained temporal reasoning capabilities. We apply this data to align Qwen2.5-Omni. The resulting model, Qwen-Audio-AHA, achieves a 13.7% improvement on AHA-Eval. Crucially, this benefit generalizes beyond our diagnostic set. Our model shows substantial gains on public benchmarks, including 1.3% on MMAU-Test and 1.6% on MMAR, outperforming latest SOTA methods. The model and dataset are open-sourced at https://github.com/LLM-VLM-GSL/AHA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。