用强化学习让大模型智能判断何时用内部知识、何时搜外部，减少无效搜索。
AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning
- 分两阶段设计，把解题和是否搜索分开决策，更清晰可控
- 实验显示搜索次数大幅减少，问答准确率损失很小
- 适合需要高效调用外部信息的实用型AI系统使用
通过强化学习为大语言模型配备搜索引擎，可构建高效的搜索代理。然而，如何自适应地平衡模型内部参数化知识与外部搜索仍具挑战：过度依赖搜索会带来额外开销并暴露于噪声或恶意内容，而仅依赖内部知识则可能导致幻觉。以往方法通过工具调用奖励工程缓解搜索滥用，但需复杂奖励设计且难以区分必要与非必要的搜索行为。为此，我们基于F1指标重新评估搜索代理，发现先前方法常忽视模型已有的内部知识。受此启发，提出AdaSearch——一种简单、两阶段、以结果为导向的强化学习框架，将问题求解与搜索决策解耦，使决策过程显式且可解释。大量实验表明，AdaSearch显著提升了搜索决策质量，大幅减少不必要的搜索调用，仅在问答准确率上相较始终搜索策略有微小下降。
原文摘要 · Abstract (English)
Equipping large language models (LLMs) with search engines via reinforcement learning (RL) promises effective search agents. However, adaptively balancing internal parametric knowledge with external search remains a challenge, as overreliance on search introduces unnecessary cost and risks exposure to noisy or malicious content, while relying solely on parametric knowledge risks hallucination. Prior efforts mitigate search overuse through tool-call reward shaping, which requires heavy reward engineering and conflates necessary and unnecessary search. To address these limitations, we revisit the evaluation of search agents through an F1-based decision metric, revealing that prior methods often overlook readily available parametric knowledge. Motivated by this, we propose AdaSearch, a simple two-stage, outcome-driven RL framework that disentangles problem-solving from the decision to search, making the decision process explicit and interpretable. Extensive experiments demonstrate that AdaSearch significantly improves search-decision quality and reduces unnecessary search calls, with only a small trade-off in QA accuracy relative to always-search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。