arXiv:2510.00568cs.CL2025-10被引 9

让搜索智能体自动纠错,提升复杂任务的准确率。

ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards

  • 引入动态纠错机制,允许智能体中途重规划搜索路径。
  • 在新基准上任务成功率超现有方法,路径忠实度显著提升。
  • 适合需要高可靠性推理的AI搜索系统开发者参考。

由大语言模型驱动的搜索智能体在处理知识密集型任务方面展现出巨大潜力。强化学习(RL)已成为训练这些智能体完成复杂多步推理的强大范式。然而,以往基于RL的方法常依赖稀疏或规则奖励,导致智能体一旦进入错误推理路径便无法恢复。为此,我们提出ReSeek,一种新型自纠正框架。该框架引入自纠正机制,使智能体能在会话过程中动态识别并修正错误搜索路径。通过调用特殊JUDGE动作,智能体可评估信息并重新规划搜索策略。为指导此过程,我们设计了稠密且具指导性的过程奖励函数,分解为事实正确性奖励和信息实用性奖励。此外,为缓解现有数据集的数据污染风险,我们提出了FictionalHot,一个新且具有挑战性的基准,包含近期精心构建、需复杂推理的问题。实验证明,使用ReSeek训练的智能体在任务成功率和路径忠实度上均显著优于现有最优基线。

原文摘要 · Abstract (English)

Search agents powered by Large Language Models (LLMs) have demonstrated significant potential in tackling knowledge-intensive tasks. Reinforcement learning (RL) has emerged as a powerful paradigm for training these agents to perform complex, multi-step reasoning. However, prior RL-based methods often rely on sparse or rule-based rewards, which can lead agents to commit to suboptimal or erroneous reasoning paths without the ability to recover. To address these limitations, we propose ReSeek, a novel self-correcting framework for training search agents. Our framework introduces a self-correction mechanism that empowers the agent to dynamically identify and recover from erroneous search paths during an episode. By invoking a special JUDGE action, the agent can judge the information and re-plan its search strategy. To guide this process, we design a dense, instructive process reward function, which decomposes into a correctness reward for retrieving factual information and a utility reward for finding information genuinely useful for the query. Furthermore, to mitigate the risk of data contamination in existing datasets, we introduce FictionalHot, a new and challenging benchmark with recently curated questions requiring complex reasoning. Being intuitively reasonable and practically simple, extensive experiments show that agents trained with ReSeek significantly outperform SOTA baselines in task success rate and path faithfulness.

搜索智能体强化学习自纠错

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。