破解大模型检索漏洞,用隐藏指令劫持系统行为
Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems
- 将恶意内容拆分为必被检索的触发片段和攻击指令
- 0.21美元即可实现近100%恶意内容检索成功率
- 首次在真实场景下完成端到端攻击,适用于各类智能系统
大型语言模型日益依赖从外部语料库中检索信息,这带来了新的攻击面:间接提示注入(IPI),即在语料库中植入隐藏指令,一旦被检索便操控模型行为。以往研究虽指出该风险,但常回避最难点——确保恶意内容实际被检索。实践中,未优化的IPI在自然查询下极少被命中,其真实影响不明。本文通过将恶意内容分解为保证被检索的触发片段与编码任意攻击目标的攻击片段,设计出高效黑盒攻击算法,仅需嵌入模型API访问权限,成本极低(使用OpenAI嵌入模型每目标用户查询仅需0.21美元),在11个基准测试和8种嵌入模型(含开源与专有服务)上实现接近100%的检索率。基于此攻击,我们首次在自然查询和真实外部语料库下实现端到端IPI攻击,涵盖RAG与代理系统,攻击目标多样。实验显示,当用户查询总结常见问题邮件时,单个被污染邮件即可使GPT-4o在多代理流程中以超过80%成功率窃取SSH密钥。我们进一步评估多种防御措施,发现均无法有效阻止恶意文本检索,凸显检索环节是关键开放漏洞。
原文摘要 · Abstract (English)
Large language models (LLMs) increasingly rely on retrieving information from external corpora. This creates a new attack surface: indirect prompt injection (IPI), where hidden instructions are planted in the corpora and hijack model behavior once retrieved. Previous studies have highlighted this risk but often avoid the hardest step: ensuring that malicious content is actually retrieved. In practice, unoptimized IPI is rarely retrieved under natural queries, which leaves its real-world impact unclear. We address this challenge by decomposing the malicious content into a trigger fragment that guarantees retrieval and an attack fragment that encodes arbitrary attack objectives. Based on this idea, we design an efficient and effective black-box attack algorithm that constructs a compact trigger fragment to guarantee retrieval for any attack fragment. Our attack requires only API access to embedding models, is cost-efficient (as little as $0.21 per target user query on OpenAI's embedding models), and achieves near-100% retrieval across 11 benchmarks and 8 embedding models (including both open-source models and proprietary services). Based on this attack, we present the first end-to-end IPI exploits under natural queries and realistic external corpora, spanning both RAG and agentic systems with diverse attack objectives. These results establish IPI as a practical and severe threat: when a user issued a natural query to summarize emails on frequently asked topics, a single poisoned email was sufficient to coerce GPT-4o into exfiltrating SSH keys with over 80% success in a multi-agent workflow. We further evaluate several defenses and find that they are insufficient to prevent the retrieval of malicious text, highlighting retrieval as a critical open vulnerability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。