通过降低模型不确定性，提升智能体搜索效率。
Search Wisely: Mitigating Sub-optimal Agentic Searches By Reducing Uncertainty
- 用强化学习引入置信度阈值，鼓励高确定性搜索决策
- 在7个问答数据集上平均准确率提升4%，30亿模型表现更优
- 揭示了搜索过载与不足源于模型对知识边界的不确定
智能体增强型检索生成系统通过多步推理和动态检索提升大语言模型能力，但常出现搜索过度（冗余信息）或搜索不足（遗漏必要信息）等问题，影响效率与可靠性。本文首次形式化定义并量化此类行为，在多个问答数据集和智能体RAG系统中发现其普遍性（例如某模型可避免27.7%的搜索步骤）。进一步揭示响应准确率与模型对自身知识边界的不确定性密切相关。为此提出β-GRPO方法，基于强化学习引入置信度阈值，奖励高确定性搜索决策。在7个问答基准上的实验表明，该方法使3B模型的智能体生成能力显著优于其他强基线，平均精确匹配分数提高4%。
原文摘要 · Abstract (English)
Agentic Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by enabling dynamic, multi-step reasoning and information retrieval. However, these systems often exhibit sub-optimal search behaviors like over-search (retrieving redundant information) and under-search (failing to retrieve necessary information), which hinder efficiency and reliability. This work formally defines and quantifies these behaviors, revealing their prevalence across multiple QA datasets and agentic RAG systems (e.g., one model could have avoided searching in 27.7% of its search steps). Furthermore, we demonstrate a crucial link between these inefficiencies and the models' uncertainty regarding their own knowledge boundaries, where response accuracy correlates with model's uncertainty in its search decisions. To address this, we propose $β$-GRPO, a reinforcement learning-based training method that incorporates confidence threshold to reward high-certainty search decisions. Experiments on seven QA benchmarks show that $β$-GRPO enable a 3B model with better agentic RAG ability, outperforming other strong baselines with a 4% higher average exact match score.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。