arXiv:2605.10235cs.CL2026-05

让大模型先判断该用检索还是长文本,提升问答效率与准确性。

Route Before Retrieve: Activating Latent Routing Abilities of LLMs for RAG vs. Long-Context Selection

论文配图:Route Before Retrieve: Activating Latent Routing Abilities of LLMs for RAG vs. Long-Context Selection
图 1 · 摘自论文原文
  • 先分析任务需求再决定使用检索或长上下文,主动规划路径。
  • 在两个数据集上表现优于传统方法,成本更低且效果更好。
  • 可迁移到小模型,适合需要高效推理的应用场景。

大语言模型上下文窗口已扩展至128K以上,支持长文档理解与多源推理。然而,如何在检索增强生成(RAG)与长上下文(LC)策略间选择仍是挑战:RAG高效但依赖检索质量,LC支持全局推理但代价高且对位置敏感。现有方法如Self-Route采用失败后退机制,被动且低效。我们提出Pre-Route,一种主动路由框架,在回答前进行结构化推理。利用轻量级元数据(如文档类型、长度、首段),实现任务分析、覆盖度估计与信息需求预测,生成可解释且低成本的路由决策。研究发现:(i) LLM具备可被引导激发的潜在路由能力,单样本性能接近多样本(Best-of-N)结果;(ii) 线性探测显示,结构化提示增强了表征空间中“最优路由维度”的可分性;(iii) 通过蒸馏可将此推理结构迁移至小模型以轻量化部署。在LaRA(域内)和LongBench-v2(域外)上的实验表明,Pre-Route优于Always-RAG、Always-LC和Self-Route基线,整体具备更优成本效益。

原文摘要 · Abstract (English)

Recent advances in large language models (LLMs) have expanded the context window to beyond 128K tokens, enabling long-document understanding and multi-source reasoning. A key challenge, however, lies in choosing between retrieval-augmented generation (RAG) and long-context (LC) strategies: RAG is efficient but constrained by retrieval quality, while LC supports global reasoning at higher cost and with position sensitivity. Existing methods such as Self-Route adopt failure-driven fallback from RAG to LC, but remain passive, inefficient, and hard to interpret. We propose Pre-Route, a proactive routing framework that performs structured reasoning before answering. Using lightweight metadata (e.g., document type, length, initial snippet), Pre-Route enables task analysis, coverage estimation, and information-need prediction, producing explainable and cost-efficient routing decisions. Our study shows three key findings: (i) LLMs possess latent routing ability that can be reliably elicited with guidelines, allowing single-sample performance to approach that of multi-sample (Best-of-N) results; (ii) linear probes reveal that structured prompts sharpen the separability of the "optimal routing dimension" in representation space; and (iii) distillation transfers this reasoning structure to smaller models for lightweight deployment. Experiments on LaRA (in-domain) and LongBench-v2 (OOD) confirm that Pre-Route outperforms Always-RAG, Always-LC, and Self-Route baselines, achieving superior overall cost-effectiveness.

RAG长文本路由决策轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。