黑盒攻击伪造文档操控大模型生成,成功率超90%。
CtrlRAG: Black-box Document Poisoning Attacks for Retrieval-Augmented Generation of Large Language Models
- 两阶段黑盒攻击:构造恶意文档并优化其检索优先级。
- 仅5个恶意文档即实现90%攻击成功率,优于基线30%。
- 适合研究安全漏洞与防御机制的学者或工程师。
检索增强生成(RAG)系统通过展示参考上下文提升回答可信度,但透明性也带来了新的黑盒攻击路径。现有文档投毒攻击多依赖不切实际的白盒或灰盒假设,实用性受限。为此,我们提出CtrlRAG,一种两阶段黑盒攻击:(1) 构造含误导或情绪诱导内容的恶意文档并注入知识库;(2) 通过定位算法与掩码语言模型(MLM)结合参考上下文反馈进行迭代优化,确保恶意文档被优先检索且语言自然。在百万文档的MS MARCO数据集上,每目标问题仅注入5个恶意文档,即可在商业大模型(如GPT-4o)上实现高达90%的攻击成功率,较最优基线提升30%,涵盖“情绪操纵”与“幻觉放大”任务。此外,现有防御措施无法兼顾安全与性能。我们提出基于参数/非参数记忆对抗的动态知识扩展防御策略,可阻断78%攻击,同时保持95.5%系统准确率。研究揭示了RAG系统的严重漏洞,并提供了有效应对方案。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems enhance response credibility and traceability by displaying reference contexts, but this transparency simultaneously introduces a novel black-box attack vector. Existing document poisoning attacks, where adversaries inject malicious documents into the knowledge base to manipulate RAG outputs, rely primarily on unrealistic white-box or gray-box assumptions, limiting their practical applicability. To address this gap, we propose CtrlRAG, a two-stage black-box attack that (1) constructs malicious documents containing misinformation or emotion-inducing content and injects them into the knowledge base, and (2) iteratively optimizes them using a localization algorithm and Masked Language Model (MLM) guided on reference context feedback, ensuring their retrieval priority while preserving linguistic naturalness. With only five malicious documents per target question injected into the million-document MS MARCO dataset, CtrlRAG achieves up to 90% attack success rates on commercial LLMs (e.g., GPT-4o), a 30% improvement over optimal baselines, in both *Emotion Manipulation* and *Hallucination Amplification* tasks. Furthermore, we show that existing defenses fail to balance security and performance. To mitigate this challenge, we introduce a dynamic *Knowledge Expansion* defense strategy based on *Parametric/Non-parametric Memory Confrontation*, blocking 78% of attacks while maintaining 95.5% system accuracy. Our findings reveal critical vulnerabilities in RAG systems and provide effective defense strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。