分离动作推理与执行授权，防止工具输出变“命令”导致失控。
When Tool Outputs Become Commands: Separating Action Induction from Runtime Authorization in Tool-Augmented LLM Agents
- 将动作生成与执行权限分离开，避免工具输出直接触发行动。
- 在多个评测中将错误执行率控制在0.63%以下，同时保持任务完成能力。
- 适合关注AI代理安全、防止意外操作的研究者与开发者。
工具增强型大模型代理在完成开放任务时依赖不可信的运行时观测；然而当工具输出不再仅提供数据，而是明确指定具体操作时，它们实际上已成为能引发真实世界副作用的“命令”，超出用户意图。我们认为这一风险源于将动作推理与执行授权混淆。为此，我们提出SARA，将动作推理与执行授权视为独立的运行时角色，并分离动作来源与执行权限。在观测端，通过上下文隔离的动作探测器揭示动作诱导语义，并持续记录跨步骤的动作来源证据作为审查信号；在执行端，工具调用仅基于用户目标和经审计的成功执行证据进行授权，同时满足目标、执行链及参数层面的支持。为保障多步执行中的分离性，SARA采用无历史提升机制，防止历史记录污染动作来源的可信性。在AgentDojo与AgentDyn上，SARA在四个主要评估场景中将误执行率（ASR）控制在不超过0.63%，同时保持竞争力的任务实用性，并在多种代理架构中一致降低ASR。
原文摘要 · Abstract (English)
Tool-augmented LLM agents must rely on untrusted runtime Observations to complete open-ended tasks; however, when tool outputs no longer merely provide data but begin to specify concrete actions, they effectively become ``commands'' that can drive real-world side effects beyond user intent. We argue that this risk arises from conflating action induction with execution authorization. To address this distinction, we propose SARA, which treats action induction and execution authorization as distinct runtime roles and separates action provenance from execution authority. On the Observation side, a context-isolated Action Probe exposes action-inducing semantics and persistently records action-origin provenance across steps as a review signal; on the execution side, actual tool calls are authorized only against the user objective and audited evidence from authorized successful executions, while satisfying goal, execution-chain, and argument-level support. To preserve this separation across multi-step execution, SARA applies No-History-Promotion to prevent historical recurrence from laundering action origins into execution authority. Across AgentDojo and AgentDyn, SARA limits ASR to no more than \(0.63\%\) across four primary evaluation settings while maintaining competitive task utility, and consistently reduces ASR across additional Agent backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。