提出新攻击方法，让大模型防御失效且仍能成功操控。
How Not to Detect Prompt Injections with an LLM
- 设计自适应攻击DataFlip，利用防御机制本身漏洞
- 0%检测率，91%攻击成功率，无需白盒访问
- 揭示现有防御的结构性缺陷，适合安全研究者
集成大模型的应用与智能体易受提示注入攻击，攻击者通过在看似无害的数据中嵌入恶意指令来操纵模型行为。近期基于已知答案检测（KAD）的防御方案声称可近乎完美地识别输入数据是否被污染。KAD试图将模型对提示注入的敏感性转化为防御能力。本文形式化分析了KAD机制，揭示其核心安全假设存在结构性漏洞。为利用这一根本缺陷，我们系统设计了一种自适应攻击——DataFlip，该攻击能持续绕过KAD防御，在不需白盒访问或优化过程的前提下，实现0%的检测率和91%的攻击成功率。
原文摘要 · Abstract (English)
LLM-integrated applications and agents are vulnerable to prompt injection attacks, where adversaries embed malicious instructions within seemingly benign input data to manipulate the LLM's intended behavior. Recent defenses based on known-answer detection (KAD) scheme have reported near-perfect performance by observing an LLM's output to classify input data as clean or contaminated. KAD attempts to repurpose the very susceptibility to prompt injection as a defensive mechanism. We formally characterize the KAD scheme and uncover a structural vulnerability that invalidates its core security premise. To exploit this fundamental vulnerability, we methodically design an adaptive attack, DataFlip. It consistently evades KAD defenses, achieving detection rates as low as $0\%$ while reliably inducing malicious behavior with a success rate of $91\%$, all without requiring white-box access to the LLM or any optimization procedures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。