arXiv:2608.08027cs.CRcs.LG2026-08

提出BASIS防御机制,精准识别可被劫持的提示注入,减少误拒。

BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes

论文配图:BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes
图 1 · 摘自论文原文
  • 用注意力竞争比ρ训练双探测器,判断注入是否存在及是否已突破
  • 在六种模型上测试,对安全攻击样本的误拒率降低超过60%
  • 适合需要高可用性的大模型应用,如客服、内容审核系统

提示注入是大语言模型应用中的关键安全威胁,攻击者通过在用户或外部数据中嵌入恶意指令来操控模型行为。现有检测方法仅判断是否存在注入并直接拒绝响应,忽略了现代对齐模型对精心设计指令具有较强鲁棒性的事实,导致不同指令和模型间的鲁棒性差异被忽视,引发广泛不必要的过量拒绝:一些模型本可正确处理的注入输入也被错误拒绝。为解决此问题,我们提出BASIS(Robustness-Aware Prompt Injection Defense),利用注意力竞争比(ρ)作为特征,训练两个稀疏线性探针——存在探测器与突破探测器。二者通过级联门控机制做出防御决策,无需额外大模型推理。BASIS包含三个阶段:注入存在检测、样本级突破预测、指令鲁棒性评估;在线级联仅在模型实际被攻破时拒绝,从而避免对鲁棒指令的误拒。在四个任务和六种开源大模型上的实验表明,BASIS保持近完美的注入检测率的同时,显著降低了对安全攻击样本的过量拒绝,尤其在强鲁棒性指令模板下表现更优。

原文摘要 · Abstract (English)

Prompt injection is a critical security threat in large language model (LLM) applications, where attackers hijack model behavior by embedding malicious instructions in user or external data. Existing detection methods only detect the presence of injection and refuse to respond upon detection, overlooking the fact that for many modern aligned models, well-crafted instructions can resist most injection attacks. This means that the injection robustness varies significantly across instructions and models. This leads to widespread unnecessary over-refusal: inputs containing injections that the model could have handled correctly are rejected incorrectly. To deal with this over-refusal issue, we propose BASIS (Robustness-Aware Prompt Injection Defense). This defense method uses the Attention Competition Ratio ($ρ$) as features to train two sparse linear probes: an existence probe and a breach probe. Both probes make defense decisions through cascaded gating, which does not require additional LLM inference. BASIS comprises three stages: injection existence detection, per-sample breach prediction, and instruction robustness assessment; the online cascade refuses only when the model would actually be compromised and thus avoids over-refusal on robust instructions. Experiments across four tasks and six open-source LLMs show that BASIS maintains near-perfect injection detection while substantially reducing over-refusal on safe attack samples, especially under robust instruction templates.

安全防御提示注入大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。