首个评估大模型检索增强生成安全性的基准，揭示其易受多种攻击影响。
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model
- 提出四类攻击：银噪声、上下文冲突、软广告、白盒拒绝服务
- 14个RAG组件在攻击下均出现服务质量下降，部分攻击可绕过现有防御
- 适合关注LLM安全与可信AI的研究者和工程师
检索增强生成（RAG）通过引入外部知识，在知识密集型任务中表现优异。然而，外部未验证知识的引入增加了大语言模型（LLMs）的脆弱性，攻击者可通过操纵知识实施攻击。本文提出SafeRAG基准，用于评估RAG安全性。首先，将攻击任务分为银噪声、上下文冲突、软广告和白盒拒绝服务四类；其次，手工构建了涵盖各类攻击的SafeRAG数据集；最后，利用该数据集模拟多种攻击场景。在14个代表性RAG组件上的实验表明，所有攻击任务均能显著破坏RAG性能，甚至最明显的攻击也能轻易绕过现有检索器、过滤器或先进LLMs，导致服务质量下降。代码已开源。
原文摘要 · Abstract (English)
The indexing-retrieval-generation paradigm of retrieval-augmented generation (RAG) has been highly successful in solving knowledge-intensive tasks by integrating external knowledge into large language models (LLMs). However, the incorporation of external and unverified knowledge increases the vulnerability of LLMs because attackers can perform attack tasks by manipulating knowledge. In this paper, we introduce a benchmark named SafeRAG designed to evaluate the RAG security. First, we classify attack tasks into silver noise, inter-context conflict, soft ad, and white Denial-of-Service. Next, we construct RAG security evaluation dataset (i.e., SafeRAG dataset) primarily manually for each task. We then utilize the SafeRAG dataset to simulate various attack scenarios that RAG may encounter. Experiments conducted on 14 representative RAG components demonstrate that RAG exhibits significant vulnerability to all attack tasks and even the most apparent attack task can easily bypass existing retrievers, filters, or advanced LLMs, resulting in the degradation of RAG service quality. Code is available at: https://github.com/IAAR-Shanghai/SafeRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。