测试RAG在真实、误导和混合信息下的可靠性表现
Evaluating RAG Reliability under Clean, Misleading, and Mixed Retrieval
- 设计三类检索数据：干净、污染、混合，系统评估RAG响应
- 发现当检索内容含误导信息时，生成结果错误率显著上升
- 适合关注大模型事实可靠性的研究人员与开发者
检索增强生成（RAG）通过引入外部证据来提升大语言模型（LLM）的事实准确性。但在信息混乱环境中，检索内容可能包含看似合理却错误的信息，引发对RAG系统可靠性的担忧。本文提出一种评估协议，系统测试RAG在参数化知识与上下文检索证据存在冲突时的表现，针对原本无需检索即可正确回答的事实性问题，分别在干净、被污染和混合证据条件下进行测试。所提出的分析框架结合参数覆盖机制与置信度指标，判断误导信息如何影响LLM生成过程。研究旨在揭示RAG在信息失序场景下的鲁棒性。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) is widely used to improve the factual reliability of large language models (LLMs) by grounding answers in retrieved evidence. In misinformation-rich environments, however, retrieved content may include plausible but incorrect information, raising concerns about the reliability of RAG-based information access systems. In this work, we propose an evaluation protocol to systematically test how the RAG system handles conflicts between parametric knowledge and evidence retrieved from context with varying amounts of misleading information. We target correct answers to factoid questions that the model responds to correctly, even when there is no retrieval, and use this to test the system with clean, poisoned, and mixed evidence. The proposed analytical framework combines parametric override and confidence metrics to assess when and how misleading information affects the generation process of LLMs. This study aims to provide insights into the robustness of RAG systems in information disorder scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。