构建基准测试,定位大模型智能体推理中幻觉出现的具体步骤。
AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents
- 提出自动幻觉归因任务,识别多步推理中引发幻觉的初始步骤。
- 在693条轨迹上测试,顶级模型最高仅41.1%准确率,工具使用幻觉最难识别(11.6%)。
- 适合关注大模型可靠性、可解释性与智能体安全的研究者。
基于大语言模型的智能体在多步序列推理中运行时,中间步骤产生的幻觉可能沿推理路径传播,降低整体可靠性。与单轮响应中的幻觉检测不同,多步工作流中的幻觉诊断需定位导致初始偏差的步骤。为填补这一空白,我们提出一项新研究任务:大模型智能体的自动化幻觉归因,旨在识别引发幻觉的步骤并解释原因。为此,我们引入AgentHallu,一个综合性基准,包含:(1) 覆盖7种智能体框架和5个领域的693条高质量推理轨迹;(2) 分为5类(规划、检索、推理、人机交互、工具使用)14子类的幻觉分类体系;(3) 多层级人工标注,涵盖二分类标签、幻觉责任步骤及因果解释。我们评估了13个主流模型,结果表明该任务极具挑战性,即使顶尖模型(如GPT-5、Gemini-2.5-Pro)表现也有限,最佳模型的步骤定位准确率为41.1%,其中工具使用类幻觉最难以识别,准确率仅为11.6%。我们认为AgentHallu将推动鲁棒、透明、可靠的智能体系统研究。
原文摘要 · Abstract (English)
As LLM-based agents operate over sequential multi-step reasoning, hallucinations arising at intermediate steps risk propagating along the trajectory, thus degrading overall reliability. Unlike hallucination detection in single-turn responses, diagnosing hallucinations in multi-step workflows requires identifying which step causes the initial divergence. To fill this gap, we propose a new research task, automated hallucination attribution of LLM-based agents, aiming to identify the step responsible for the hallucination and explain why. To support this task, we introduce AgentHallu, a comprehensive benchmark with: (1) 693 high-quality trajectories spanning 7 agent frameworks and 5 domains, (2) a hallucination taxonomy organized into 5 categories (Planning, Retrieval, Reasoning, Human-Interaction, and Tool-Use) and 14 sub-categories, and (3) multi-level annotations curated by humans, covering binary labels, hallucination-responsible steps, and causal explanations. We evaluate 13 leading models, and results show the task is challenging even for top-tier models (like GPT-5, Gemini-2.5-Pro). The best-performing model achieves only 41.1\% step localization accuracy, where tool-use hallucinations are the most challenging at just 11.6\%. We believe AgentHallu will catalyze future research into developing robust, transparent, and reliable agentic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。