自动优化检索器，低成本实现精准匹配
SEARA: An Automated Approach for Obtaining Optimal Retrievers
- 通过子集采样降低评估成本
- 仅用少量检索事实即完成评估
- 适合需要快速适配业务场景的团队
检索增强生成（RAG）是提升大语言模型性能的核心方法，其效果高度依赖于检索器的表现。然而，检索器存在大量超参数，对特定应用场景敏感，而超参数调优计算开销巨大。现有评估方法或成本过高，或与实际场景脱节。本文提出SEARA（基于子集采样的自动检索器评估方法），通过子集采样解决评估数据难题，仅需极少检索事实提取与全面检索指标，即可实现低代价、高鲁棒性的自动化检索器评估。基于真实用户查询，该方法可全自动完成评估，高效获取特定业务场景下的最优检索器。我们在红笔记（rednote）的典型RAG应用中验证了该方法，包括基于知识的问答系统和基于检索的旅行助手，成功获得场景定制化最优检索器。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) is a core approach for enhancing Large Language Models (LLMs), where the effectiveness of the retriever largely determines the overall response quality of RAG systems. Retrievers encompass a multitude of hyperparameters that significantly impact performance outcomes and demonstrate sensitivity to specific applications. Nevertheless, hyperparameter optimization entails prohibitively high computational expenses. Existing evaluation methods suffer from either prohibitive costs or disconnection from domain-specific scenarios. This paper proposes SEARA (Subset sampling Evaluation for Automatic Retriever Assessment), which addresses evaluation data challenges through subset sampling techniques and achieves robust automated retriever evaluation by minimal retrieval facts extraction and comprehensive retrieval metrics. Based on real user queries, this method enables fully automated retriever evaluation at low cost, thereby obtaining optimal retriever for specific business scenarios. We validate our method across classic RAG applications in rednote, including knowledge-based Q\&A system and retrieval-based travel assistant, successfully obtaining scenario-specific optimal retrievers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。