小模型通过自我反思和迭代搜索,实现高效精准检索。
Think Before You Retrieve: Learning Test-Time Adaptive Search with Small Language Models
- 让小模型学会边查边想、动态调整策略
- 1.2B模型在多个数据集上超越大模型,最高提升5%以上
- 适合资源有限但需智能检索的场景
有效信息检索需要基于部分证据进行推理,并随信息出现不断优化策略。当前方法存在局限:神经检索器缺乏推理能力,大语言模型虽具语义深度但成本过高,查询重写或分解仅支持静态变换。为此,我们提出Orion框架,使参数量350M-1.2B的小模型可通过学习到的搜索策略实现迭代式检索。该框架结合:(1)合成轨迹生成与监督微调,鼓励多样化探索;(2)强化学习,奖励有效查询优化与回溯行为;(3)推理时束搜索算法,利用强化学习中习得的自我反思能力。尽管仅使用3%的训练数据,1.2B模型在SciFact上达到77.6%成功率(前代为72.6%),BRIGHT达25.2%(前代22.1%),NFCorpus达63.2%(前代57.8%),在FEVER、HotpotQA、MSMarco上也保持竞争力。其性能优于规模高达200-400倍的检索器,在六项基准中的五项表现更优。结果表明,当模型被训练为能搜索、反思与修正时,检索性能可超越模型规模限制。
原文摘要 · Abstract (English)
Effective information retrieval requires reasoning over partial evidence and refining strategies as information emerges. Yet current approaches fall short: neural retrievers lack reasoning capabilities, large language models (LLMs) provide semantic depth but at prohibitive cost, and query rewriting or decomposition limits improvement to static transformations. As a result, existing methods fail to capture the iterative dynamics of exploration, feedback, and revision that complex user queries demand. We introduce Orion, a training framework that enables compact models (350M-1.2B parameters) to perform iterative retrieval through learned search strategies. Orion combines: (1) synthetic trajectory generation and supervised fine-tuning to encourage diverse exploration patterns in models, (2) reinforcement learning (RL) that rewards effective query refinement and backtracking behaviors, and (3) inference-time beam search algorithms that exploit the self-reflection capabilities learned during RL. Despite using only 3% of the training data available, our 1.2B model achieves 77.6% success on SciFact (vs. 72.6% for prior retrievers), 25.2% on BRIGHT (vs. 22.1%), 63.2% on NFCorpus (vs. 57.8%), and remains competitive on FEVER, HotpotQA, and MSMarco. It outperforms retrievers up to 200-400x larger on five of six benchmarks. These findings suggest that retrieval performance can emerge from learned strategies, not just model scale, when models are trained to search, reflect, and revise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。