arXiv:2602.11551cs.CL2026-02被引 2

SIGHT让语言模型搜索更智能,减少重复信息,提升准确率。

SIGHT: Reinforcement Learning with Self-Evidence and Information-Gain Diverse Branching for Search Agent

  • 用自证据提炼和信息增益引导分支,优化搜索路径。
  • 在多跳问答中以更少步骤达到更高准确率。
  • 适合需要高效推理的复杂问题求解场景。

强化学习已使大语言模型具备自主搜索解决复杂问答的能力。但在多轮搜索中,结果常出现高冗余与低信噪比,导致代理陷入‘隧道视觉’,早期噪声检索引发错误累积。为此,我们提出SIGHT框架,通过自证据支持(SES)提炼高质量证据,并计算信息增益分数,识别能最大降低不确定性的关键状态。该分数驱动动态提示干预——包括去重、反思或自适应分支——生成带SES的新分支。最终,通过组相对策略优化整合SES与正确性奖励,使SIGHT在无外部验证下内化稳健探索策略。在单跳与多跳问答基准上实验表明,SIGHT显著优于现有方法,尤其在复杂推理场景中,且使用更少搜索步数。

原文摘要 · Abstract (English)

Reinforcement Learning (RL) has empowered Large Language Models (LLMs) to master autonomous search for complex question answering. However, particularly within multi-turn search scenarios, this interaction introduces a critical challenge: search results often suffer from high redundancy and low signal-to-noise ratios. Consequently, agents easily fall into "Tunnel Vision," where the forced interpretation of early noisy retrievals leads to irreversible error accumulation. To address these challenges, we propose SIGHT, a framework that enhances search-based reasoning through Self-Evidence Support (SES) and Information-Gain Driven Diverse Branching. SIGHT distills search results into high-fidelity evidence via SES and calculates an Information Gain score to pinpoint pivotal states where observations maximally reduce uncertainty. This score guides Dynamic Prompting Interventions - including de-duplication, reflection, or adaptive branching - to spawn new branches with SES. Finally, by integrating SES and correctness rewards via Group Relative Policy Optimization, SIGHT internalizes robust exploration strategies without external verifiers. Experiments on single-hop and multi-hop QA benchmarks demonstrate that SIGHT significantly outperforms existing approaches, particularly in complex reasoning scenarios, using fewer search steps.

强化学习搜索推理大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。