arXiv:2606.30531cs.AI2026-06被引 1

研究工具增强型智能体中实体绑定错误,揭示其对安全性的威胁

Entity Binding Failures in Tool-Augmented Agents

  • 区分工具选择正确性与实体绑定正确性,提出错误分类体系
  • 实验显示24%-26%的运行出现错误实体操作,即使工具选择正确
  • 引入实体感知机制可消除错误,但可能延迟任务完成

工具增强的语言模型智能体通常通过是否选择正确工具、生成有效API参数及完成任务来评估。然而,智能体可能选择了正确工具,却作用于错误的外部实体。例如,“给亚历克斯发邮件关于发布”可能导致联系错误的亚历克斯、附上错误的发布文档、回复错误的邮件线程或更新错误的客户账户。此类错误称为实体绑定失败。本文将实体绑定失败视为工具增强智能体中独立的可靠性与安全性问题。我们形式化了工具正确性与实体正确性的分离,提出企业工作流中错误实体行为的分类体系,并评估了包括实体解析前置条件、置信度门控绑定、模糊情况下的澄清以及溯源追踪在内的实体感知执行机制。在60个任务、五个模型后端和六种工具使用方法的控制诊断评估中,所有方法均实现0.0%的错误工具选择率,但行动导向基线仍导致24.0%-26.0%的运行出现错误实体操作。实体感知方法在该设置下消除了错误实体操作及风险加权的错误实体暴露,但因在模糊情况下推迟决策而降低了直接任务完成率。结果表明,安全的工具使用不仅需要选择正确工具,还需在执行前可靠地将自然语言指代绑定到正确的现实世界实体。

原文摘要 · Abstract (English)

Tool-augmented language-model agents are often evaluated by whether they select the correct tool, produce valid API arguments, and complete the requested task. However, an agent may choose the right tool and still act on the wrong external entity. For example, a request to "email Alex about the launch" may lead the agent to contact the wrong Alex, attach the wrong launch document, reply in the wrong thread, or update the wrong customer account. We call these errors entity binding failures. This paper studies entity binding failures as a distinct reliability and safety problem in tool-augmented agents. We formalize the separation between tool correctness and entity correctness, introduce a taxonomy of wrong-entity failures in enterprise workflows, and evaluate entity-aware execution mechanisms including entity-resolution preconditions, confidence-gated binding, clarification under ambiguity, and provenance tracking. In a controlled diagnostic evaluation across 60 tasks, five model backends, and six tool-use methods, all methods achieved 0.0 percent wrong-tool error, yet action-oriented baselines still produced wrong-entity actions in 24.0-26.0 percent of runs. Entity-aware methods eliminated wrong-entity actions and risk-weighted wrong-entity exposure in this setting, but reduced direct task completion by deferring under ambiguity. These findings show that safe tool use requires not only selecting the correct tool, but also reliably binding natural-language references to the correct real-world entity before action.

智能体实体绑定安全工具使用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。