系统评测RAG核心组件，找短板提性能
XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation
- 分预检索、检索、后处理、生成四阶段评测组件
- 在重构数据集上验证各阶段表现，定位失效点
- 开源工具链支持故障诊断与优化，适合研发者使用
检索增强生成（RAG）通过结合外部数据检索与大语言模型的生成能力，确保输出既相关又准确。本文提出XRAG，一个开源、模块化的代码库，用于全面评估先进RAG系统中基础组件的性能。这些组件被系统性地划分为四个核心阶段：预检索、检索、后处理和生成。我们在重构的数据集上对各阶段进行系统分析，提供全面的基准测试。随着RAG系统复杂度持续提升，识别潜在失效点变得至关重要。为此，我们设计了一套实验方法和诊断测试协议，用于剖析RAG工程中的固有缺陷，并提出针对性解决方案以提升整体性能。本工作深入评估了RAG核心组件的表现，为常见失效点的优化提供了重要洞察。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) synergizes the retrieval of pertinent data with the generative capabilities of Large Language Models (LLMs), ensuring that the generated output is not only contextually relevant but also accurate and current. We introduce XRAG, an open-source, modular codebase that facilitates exhaustive evaluation of the performance of foundational components of advanced RAG modules. These components are systematically categorized into four core phases: pre-retrieval, retrieval, post-retrieval, and generation. We systematically analyse them across reconfigured datasets, providing a comprehensive benchmark for their effectiveness. As the complexity of RAG systems continues to escalate, we underscore the critical need to identify potential failure points in RAG systems. We formulate a suite of experimental methodologies and diagnostic testing protocols to dissect the failure points inherent in RAG engineering. Subsequently, we proffer bespoke solutions aimed at bolstering the overall performance of these modules. Our work thoroughly evaluates the performance of advanced core components in RAG systems, providing insights into optimizations for prevalent failure points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。