用大模型内部表示实时检测工具调用幻觉，提升智能体可靠性。
Internal Representations as Indicators of Hallucinations in Agent Tool Selection
- 利用生成时的内部表示，单次前向传播实现幻觉检测。
- 在多领域推理任务中最高达86.4%检测准确率，尤其擅长识别参数错误。
- 适合需要高可靠性的生产级智能体系统部署。
大型语言模型（LLMs）在工具调用与使用方面表现出色，但存在选择错误工具、参数格式错误及‘绕过工具’行为（即直接模拟生成而非调用专用工具或外部系统），这削弱了基于LLM的智能体在生产环境中的可靠性，导致结果不一致，并规避安全与审计控制。此类工具选择幻觉需早期检测与处理。现有方法多依赖多次前向传播或外部验证，计算开销大。本文提出一种计算高效框架，在生成同一前向传播过程中，利用LLM内部表示实现工具调用幻觉的实时检测。我们在多个领域的推理任务上评估该方法，结果显示检测性能优异（最高86.4%准确率），同时保持实时推理能力且计算开销极低，尤其在检测参数级幻觉和不当工具选择方面表现突出，对智能体的可靠部署至关重要。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown remarkable capabilities in tool calling and tool usage, but suffer from hallucinations where they choose incorrect tools, provide malformed parameters and exhibit 'tool bypass' behavior by performing simulations and generating outputs instead of invoking specialized tools or external systems. This undermines the reliability of LLM based agents in production systems as it leads to inconsistent results, and bypasses security and audit controls. Such hallucinations in agent tool selection require early detection and error handling. Unlike existing hallucination detection methods that require multiple forward passes or external validation, we present a computationally efficient framework that detects tool-calling hallucinations in real-time by leveraging LLMs' internal representations during the same forward pass used for generation. We evaluate this approach on reasoning tasks across multiple domains, demonstrating strong detection performance (up to 86.4\% accuracy) while maintaining real-time inference capabilities with minimal computational overhead, particularly excelling at detecting parameter-level hallucinations and inappropriate tool selections, critical for reliable agent deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。