攻击者通过伪造指令提示,悄悄操控检索增强生成系统输出。
AIP: Subverting Retrieval-Augmented Generation via Adversarial Instructional Prompt
- 用遗传算法优化恶意指令提示,隐蔽改变检索结果
- 攻击成功率高达95.23%,同时保持提示正常功能
- 揭示公开共享提示的隐蔽风险,适合安全研究者关注
检索增强生成(RAG)通过从外部源检索相关文档来提升大语言模型的事实准确性和可验证性。然而,这种依赖引入了新的攻击面,尤其在检索管道中。现有攻击多依赖操纵用户查询,但实际中用户输入常受保护或固定,难以操作。本文提出一种新型攻击——对抗性指令提示(AIP),将攻击目标转向广泛复用、公开共享且极少审计的指令提示。这些看似无害的接口组件因被默认信任,成为隐蔽操控RAG行为的理想载体。AIP通过微妙修改检索行为,实现三个目标:自然性(避免被察觉)、实用性(鼓励使用)和鲁棒性(适应不同查询变体)。我们设计了一种模拟真实语言变异的多样化查询生成策略,以发现能泛化到改写和重述的攻击提示。进一步采用基于遗传算法的联合优化方法,在攻击成功率、原始任务可用性与隐蔽性之间取得平衡。实验表明,AIP在保持良性功能的同时,最高可达95.23%的攻击成功比率(ASR)。该研究揭示了RAG系统中一个此前被忽视的关键漏洞,强调必须重新评估共享指令提示的安全性。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by retrieving relevant documents from external sources to improve factual accuracy and verifiability. However, this reliance introduces new attack surfaces within the retrieval pipeline, beyond the LLM itself. While prior RAG attacks have exposed such vulnerabilities, they largely rely on manipulating user queries, which is often infeasible in practice due to fixed or protected user inputs. This narrow focus overlooks a more realistic and stealthy vector: instructional prompts, which are widely reused, publicly shared, and rarely audited. Their implicit trust makes them a compelling target for adversaries to manipulate RAG behavior covertly. We introduce a novel attack for Adversarial Instructional Prompt (AIP) that exploits adversarial instructional prompts to manipulate RAG outputs by subtly altering retrieval behavior. By shifting the attack surface to the instructional prompts, AIP reveals how trusted yet seemingly benign interface components can be weaponized to degrade system integrity. The attack is crafted to achieve three goals: (1) naturalness, to evade user detection; (2) utility, to encourage use of prompts; and (3) robustness, to remain effective across diverse query variations. We propose a diverse query generation strategy that simulates realistic linguistic variation in user queries, enabling the discovery of prompts that generalize across paraphrases and rephrasings. Building on this, a genetic algorithm-based joint optimization is developed to evolve adversarial prompts by balancing attack success, clean-task utility, and stealthiness. Experimental results show that AIP achieves up to 95.23% ASR while preserving benign functionality. These findings uncover a critical and previously overlooked vulnerability in RAG systems, emphasizing the need to reassess the shared instructional prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。