发现智能大模型在遭遇间接提示攻击时,隐藏状态中会悄悄留下可被探测的痕迹。
Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden States

- 用线性探针从预生成隐藏状态中识别出间接攻击暴露信号
- 在跨语言和自适应攻击下仍保持0.90+的检测准确率
- 提出基于探针引导的防御机制,有效降低攻击成功率至0%
智能大模型易受间接提示注入(IPI)攻击,例如外部工具结果中隐藏的恶意任务。本文从三个角度研究了模型在遭受此类攻击时的内部状态表现。首先,通过8个模型(包括7530亿参数的GLM-5.2和2.8万亿参数的Kimi-K3)验证:仅用简单线性探针分析预生成隐藏状态,即可在未见过的攻击、指令与任务集上实现0.90以上的AUROC,且对自适应攻击和跨语言场景保持稳健。其次,揭示了后训练模型存在“知识-行为”断层:虽能编码出反映攻击暴露的信号,却无法可靠转化为安全行为;为此提出探针门控推理防御,在困难的AgentDojo设置下显著降低攻击成功率(如Qwen3.5-27B从34.6%降至0%),同时优于基线方法保留正常任务性能。最后,构建分析框架,发现自然语言解释与探针捕捉信号强相关,不同模型的潜在信号模式分别对应直接暴露感知或间接操作线索。代码已开源。
原文摘要 · Abstract (English)
Agentic LLMs are vulnerable to indirect prompt injection (IPI) attacks, e.g., malicious side-tasks hidden in external tool results. While many efforts have sought to address this threat, little is known about the internals of agentic LLMs when they are exposed to IPI attacks. For simplicity, we refer to this condition as IPI exposure. In this paper, we study IPI exposure from three perspectives. (1) Probing: Across eight models, including the 753B-parameter GLM-5.2 and the 2.8T-parameter Kimi-K3, simple linear probes trained on pre-generation hidden states can predict LLMs' IPI exposure. These probes achieve 0.90+ AUROC on unseen attacks, agent instructions, and task suites; they remain robustly predictive under adaptive attacks and in cross-lingual settings. (2) Defense: We reveal and diagnose a knowledge-action gap: post-trained LLMs encode signals predictive of IPI exposure, yet do not reliably bind these signals to safe agentic actions. We therefore introduce a probe-gated reasoning-based defense to bridge this gap at test time. On difficult AgentDojo settings, it substantially reduces attack success rate, e.g., from 34.6% to 0% on Qwen3.5-27B, and better preserves clean-task utility than the baselines. (3) Explanation: We introduce an analysis framework that identifies natural-language explanations strongly correlated with probe-captured signals. The resulting profiles differ across models: latent signals can align with either direct IPI-exposure sensing or indirect operational cues. Code is available at https://github.com/jianshuod/IPI-exposure-signal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。