用合成语义信息增益奖励优化智能体检索，提升大模型推理准确率。
Optimizing Agentic Reasoning with Retrieval via Synthetic Semantic Information Gain Reward
- 通过语义聚类与文本蕴含关系，从模型输出直接估算信息增益。
- 在7个问答基准上平均提升5.4%准确率，优于现有检索增强方法。
- 理论可证明信息增益的非负性与可加性，适合无标注数据训练。
智能体推理使大型推理模型能动态获取外部知识，但检索过程缺乏密集且合理的奖励信号，优化仍具挑战。本文提出InfoReasoner，一个统一框架，通过合成语义信息增益奖励激励有效信息获取。理论上，将信息增益重新定义为模型信念状态的不确定性降低，建立保证，包括非负性、望远镜可加性和信道单调性。实践中，为实现无需人工标注的可扩展优化，我们提出一种输出感知内在估计器，利用双向文本蕴含进行语义聚类，直接从模型输出分布计算信息增益。该内在奖励引导策略最大化认知进展，支持通过组相对策略优化（GRPO）高效训练。在七个问答基准上的实验表明，InfoReasoner持续优于强基线，平均准确率提升达5.4%。本工作为带检索的智能体推理提供了理论扎实且可扩展的路径。代码已开源：https://github.com/dl-m9/InfoReasoner。
原文摘要 · Abstract (English)
Agentic reasoning enables large reasoning models (LRMs) to dynamically acquire external knowledge, but yet optimizing the retrieval process remains challenging due to the lack of dense, principled reward signals. In this paper, we introduce InfoReasoner, a unified framework that incentivizes effective information seeking via a synthetic semantic information gain reward. Theoretically, we redefine information gain as uncertainty reduction over the model's belief states, establishing guarantees, including non-negativity, telescoping additivity, and channel monotonicity. Practically, to enable scalable optimization without manual retrieval annotations, we propose an output-aware intrinsic estimator that computes information gain directly from the model's output distributions using semantic clustering via bidirectional textual entailment. This intrinsic reward guides the policy to maximize epistemic progress, enabling efficient training via Group Relative Policy Optimization (GRPO). Experiments across seven question-answering benchmarks demonstrate that InfoReasoner consistently outperforms strong retrieval-augmented baselines, achieving up to 5.4% average accuracy improvement. Our work provides a theoretically grounded and scalable path toward agentic reasoning with retrieval. The code is available at https://github.com/dl-m9/InfoReasoner
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。