研究查询扰动对检索增强生成的影响，发现小改动就可能让系统失效
Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
- 拆解RAG各模块，测试不同查询变化下的表现
- 微小查询修改导致检索性能显著下降，部分超过30%
- 提出评估框架，为实际应用提供优化建议
大语言模型（LLMs）更新新信息成本高且效率低。为解决此问题，检索增强生成（RAG）被提出，在推理时动态引入外部知识，提升事实一致性并减少幻觉。然而，RAG系统面临实际挑战——高度依赖输入查询质量以实现准确检索。本文研究RAG流水线中各组件对各类查询扰动的敏感性。分析显示，常用检索器在轻微查询变化下性能可能大幅下降。我们在通用和领域特定数据集上，分别独立考察各模块及端到端问答场景中的联合影响，并进行了超过1092次实验。此外，我们提出一个系统化的评估框架，用于衡量RAG在查询层面的鲁棒性，并基于结果给出可操作的实践建议。
原文摘要 · Abstract (English)
Large language models (LLMs) are very costly and inefficient to update with new information. To address this limitation, retrieval-augmented generation (RAG) has been proposed as a solution that dynamically incorporates external knowledge during inference, improving factual consistency and reducing hallucinations. Despite its promise, RAG systems face practical challenges-most notably, a strong dependence on the quality of the input query for accurate retrieval. In this paper, we investigate the sensitivity of different components in the RAG pipeline to various types of query perturbations. Our analysis reveals that the performance of commonly used retrievers can degrade significantly even under minor query variations. We study each module in isolation as well as their combined effect in an end-to-end question answering setting, using both general-domain and domain-specific datasets. Additionally, we propose an evaluation framework to systematically assess the query-level robustness of RAG pipelines and offer actionable recommendations for practitioners based on the results of more than 1092 experiments we performed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。