现有防御方法容易误拒正常指令,因依赖表面模式而非真实意图。
Defenses Against Prompt Attacks Learn Surface Heuristics
- 通过监督微调学习攻击表面特征,而非深层恶意意图。
- 良性内容放末尾时被拒绝率从10%飙升至90%,触发词使误拒增50%。
- 模型在未见数据上准确率最高下降40%,适合安全研究者警惕伪防御。
大型语言模型日益应用于安全敏感场景,需遵循系统或开发者定义的任务逻辑,同时处理普通用户请求。当用户查询或外部检索内容中出现对抗性指令时,模型可能偏离预期行为。近期防御方法依赖于良性与恶意标签的监督微调,虽能实现高攻击拒绝率,但发现其依赖防御数据中的窄相关性,而非有害意图,导致对安全输入的系统性误拒。我们分析了三种由防御微调引发的捷径行为:位置偏差——良性内容置于提示末尾时被拒绝率从低于10%升至高达90%;标记触发偏差——攻击数据中常见字符串在良性上下文中也提高拒绝概率,插入一个触发标记可使误拒增加最多50%;主题泛化偏差——模型在防御数据分布外泛化能力差,测试时准确率最高下降40%。这些结果表明,当前提示注入防御常响应攻击式表面模式,而非潜在意图。我们构建了受控诊断数据集,并在两个基础模型和多个防御流程中进行系统评估,揭示监督微调在可靠LLM安全中的局限性。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly deployed in security-sensitive applications, where they must follow system- or developer-specified instructions that define the intended task behavior, while completing benign user requests. When adversarial instructions appear in user queries or externally retrieved content, models may override intended logic. Recent defenses rely on supervised fine-tuning with benign and malicious labels. Although these methods achieve high attack rejection rates, we find that they rely on narrow correlations in defense data rather than harmful intent, leading to systematic rejection of safe inputs. We analyze three recurring shortcut behaviors induced by defense fine-tuning. \emph{Position bias} arises when benign content placed later in a prompt is rejected at much higher rates; across reasoning benchmarks, suffix-task rejection rises from below \textbf{10\%} to as high as \textbf{90\%}. \emph{Token trigger bias} occurs when strings common in attack data raise rejection probability even in benign contexts; inserting a single trigger token increases false refusals by up to \textbf{50\%}. \emph{Topic generalization bias} reflects poor generalization beyond the defense data distribution, with defended models suffering test-time accuracy drops of up to \textbf{40\%}. These findings suggest that current prompt-injection defenses frequently respond to attack-like surface patterns rather than the underlying intent. We introduce controlled diagnostic datasets and a systematic evaluation across two base models and multiple defense pipelines, highlighting limitations of supervised fine-tuning for reliable LLM security.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。