RAG增强会削弱安全防护模型可靠性，11%判断出错
RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contexts
- 用RAG上下文测试防护模型，发现额外信息干扰判断
- 11%输入和8%输出判断被良性文档误导
- 现有缓解方法效果有限，需更鲁棒的训练评估
随着大语言模型（LLMs）广泛应用，确保其安全性成为紧迫问题。基于外部LLM的防护模型虽被广泛采用，但本身是微调或提示工程的LLM，易受数据分布偏移影响。本文以检索增强生成（RAG）为例，系统评估了3个Llama Guard和2个GPT-oss模型在包含额外信息的上下文中的鲁棒性。结果表明，向防护上下文中插入良性文档，使输入和输出防护判断错误率分别达约11%和8%，导致其不可靠。我们分别分析了检索文档、用户查询和LLM生成回复对防护判断的影响。两种缓解方法仅带来轻微改善。该结果揭示当前防护模型在上下文鲁棒性上的缺口，呼吁建立对检索与查询组合更具鲁棒性的训练与评估协议。
原文摘要 · Abstract (English)
With the increasing adoption of large language models (LLMs), ensuring the safety of LLM systems has become a pressing concern. External LLM-based guardrail models have emerged as a popular solution to screen unsafe inputs and outputs, but they are themselves fine-tuned or prompt-engineered LLMs that are vulnerable to data distribution shifts. In this paper, taking Retrieval Augmentation Generation (RAG) as a case study, we investigated how robust LLM-based guardrails are against additional information embedded in the context. Through a systematic evaluation of 3 Llama Guards and 2 GPT-oss models, we confirmed that inserting benign documents into the guardrail context alters the judgments of input and output guardrails in around 11% and 8% of cases, making them unreliable. We separately analyzed the effect of each component in the augmented context: retrieved documents, user query, and LLM-generated response. The two mitigation methods we tested only bring minor improvements. These results expose a context-robustness gap in current guardrails and motivate training and evaluation protocols that are robust to retrieval and query composition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。