arXiv:2608.24022cs.CRcs.AI2026-08

定位大模型推理中操控工具调用的隐藏指令,提升智能体安全性。

What Guides the Agent? Adjudicating Unauthorized Behavior via Localizing Behavior-Guiding Instructions

论文配图:What Guides the Agent? Adjudicating Unauthorized Behavior via Localizing Behavior-Guiding Instructions
图 1 · 摘自论文原文
  • 将指令定位转为注意力图中的目标检测问题,精准识别影响决策的上下文片段。
  • 在10种配置下平均检测准确率达95.6%,真阳性率93.4%且误报率仅6.7%。
  • 可跨模型迁移并动态判断指令来源可信度,适合高风险场景的智能体防护。

集成外部资源的大语言模型智能体具备复杂任务能力,但统一的自然语言上下文通道使其易受注入攻击:不可信的外部数据可能在模型推理过程中被动态解析为行为引导指令,从而扭曲智能体决策。现有防御措施主要集中在输入输出层面的静态检测或隔离,难以应对推理过程中动态产生的诱导行为。本文提出 Attnlocate,一个运行时细粒度定位行为引导指令的框架。该方法将定位问题建模为对象检测任务,通过多头多层注意力聚合构建针对目标检测优化的词元级特征空间,并采用无锚点的1-D U-Net检测器识别相关上下文片段。最后,基于检测到片段的提供方权限,动态判定恶意调用尝试。我们在五类大模型共十种代理配置下评估,涵盖间接提示注入与工具污染场景。Attnlocate 实现平均交并比0.743,平均AUROC 0.956,真阳性率93.4%且假阳性率仅0.067。模型具有强泛化能力,支持无需重训练的权限策略调整。

原文摘要 · Abstract (English)

LLM agents integrated with external resources gain complex task capabilities, yet the unified natural-language context channel makes them vulnerable to injection attacks: untrusted external data may be dynamically parsed as behavior-guiding instructions during LLM inference, thereby subverting the agent's decision. Existing defenses focus on static detection or isolation of malicious content at the input/output level, remains insufficient for detecting such dynamic inducements that arise during model reasoning. We propose Attnlocate, a runtime framework for fine-grained localization of context spans that genuinely influence tool-calling decisions, i.e., behavior-guiding instructions. Attnlocate casts this localization problem as an object detection task, aiming to detect the distinctive activation traces induced by behavior-guiding instructions within the attention matrix. Specifically, we design a multi-head, multi-layer attention aggregation scheme to construct a token-level feature space tailored for object detection. Then, a 1-D U-Net equipped with an anchor-free detection head is deployed to detect these spans. Finally, based on the authority of the provider from which the detected behavior-guiding spans originate, Attnlocate dynamically adjudicates malicious invocation attempts. We evaluate Attnlocate across ten agent configurations from five LLM families, covering scenarios involving indirect prompt injection and tool poisoning. Attnlocate achieves a mean IoU of 0.743, an average AUROC of 0.956, and a 0.934 true-positive rate at 0.067 false-positive rate. It also transfers effectively across unseen models and supports authority policy adaptation without retraining.

智能体安全指令定位注意力检测防御机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。