首个自适应RAG路由基准,帮系统选最适合的生成方案。
RAGRouter-Bench: A Dataset and Benchmark for Adaptive RAG Routing
- 基于查询-语料兼容性设计多维度评估体系
- 实测显示无通用最优方案,动态路由更高效
- 适合想优化RAG性能与成本的研究者和工程师
检索增强生成(RAG)已发展为多种具有不同性能特征和资源需求的范式,使得范式选择成为多标准、上下文依赖的决策问题。然而,现有研究大多聚焦单一方法改进或仅基于查询的评测,缺乏对RAG范式在多样化查询-语料情境下表现及效果-效率权衡的系统分析。本文提出RAGRouter-Bench,首个用于自适应RAG路由的数据集与基准。该基准基于查询-语料兼容性,整合三种典型查询类型、细粒度语料指标(涵盖结构与语义属性),并建立统一评估协议,同时衡量生成质量与资源消耗。我们对多个骨干大模型在所有查询-语料组合上标准化实现了RAG范式,构建了包含定量指标与大模型评判的综合性基准,支持上下文感知且成本可控的RAG路由决策。进一步将路由建模为上下文依赖的范式选择,并在该数据集上评测多种查询-语料路由策略。大量实验表明,不存在适用于所有查询-语料对的通用范式,而自适应路由相比固定范式选择能获得更优的效果-效率权衡。这些发现确立了查询-语料兼容性作为自适应RAG路由的核心原则,并将RAGRouter-Bench定位为下一代RAG系统的系统化测试平台。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) has evolved into a family of paradigms with distinct performance profiles and resource demands, turning paradigm selection into a multi-criteria, context-dependent decision problem. Nevertheless, existing studies largely focus on isolated method improvements or query-only benchmarking, without systematically examining how RAG paradigms behave across diverse query-corpus contexts and effectiveness-efficiency trade-offs. In this work, we introduce RAGRouter-Bench, the first dataset and benchmark for adaptive RAG routing. Grounded in query-corpus compatibility, the benchmark integrates three canonical query types, fine-grained corpus indicators capturing structural and semantic properties, and a unified protocol for evaluating both generation quality and resource consumption. Then, we implement standardized RAG paradigms with multiple backbone LLMs across all query-corpus combinations, constructing a comprehensive benchmark with quantitative metrics and LLM-as-a-Judge evaluations to inform context-aware and cost-effective RAG routing decisions. We further formulate routing as context-dependent paradigm selection and benchmark a range of query-corpus routers on the constructed dataset. Extensive experiments demonstrate that no one-size-fits-all paradigm exists across query-corpus pairs, and that adaptive routing yields more favorable effectiveness-efficiency trade-offs than fixed paradigm selection. These findings establish query-corpus compatibility as a central principle for adaptive RAG routing and position RAGRouter-Bench as a systematic testbed for next-generation RAG systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。