自动挑选最适合数据集的RAG模块，提升问答效果
AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline
- 用自动化方法搜索最优RAG模块组合
- 在多个数据集上验证性能提升效果
- 适合需要高效构建RAG系统的开发者
将大语言模型与外部文档结合已使检索增强生成（RAG）成为关键技术。尽管已有众多RAG技术与组件被研究，但其表现会因数据集不同而异。在特定数据集上找到表现优异的RAG组件极具挑战性。本文提出AutoRAG框架，可自动识别适用于给定数据集的合适RAG组件。该框架探索并近似出针对特定数据集的最优组件组合。此外，我们展示了使用AutoRAG优化数据集的实验结果。所有实验数据与代码已公开，可通过GitHub仓库 https://github.com/Marker-Inc-Korea/AutoRAG_ARAGOG_Paper 获取。
原文摘要 · Abstract (English)
Using LLMs (Large Language Models) in conjunction with external documents has made RAG (Retrieval-Augmented Generation) an essential technology. Numerous techniques and modules for RAG are being researched, but their performance can vary across different datasets. Finding RAG modules that perform well on specific datasets is challenging. In this paper, we propose the AutoRAG framework, which automatically identifies suitable RAG modules for a given dataset. AutoRAG explores and approximates the optimal combination of RAG modules for the dataset. Additionally, we share the results of optimizing a dataset using AutoRAG. All experimental results and data are publicly available and can be accessed through our GitHub repository https://github.com/Marker-Inc-Korea/AutoRAG_ARAGOG_Paper .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。