用细粒度评分标准指导智能体每一步行动,让搜索推理更精准。
Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents

- 将评分标准嵌入每步决策上下文,引导证据搜集与推理方向。
- 在多个评测集上超越传统ReAct和计算资源优化基线,提升显著。
- 评分生成器可直接接入现有模型,无需改动核心机制。
针对搜索密集型多步推理任务中,基于ReAct的智能体依赖自身判断导致轨迹浅层、冗余或目标偏差的问题,本文提出Co-ReAct框架,首次将评分标准作为推理过程中的细粒度步骤级指导信号。在每一步决策中,Co-ReAct将评分标准注入智能体上下文,明确指引下一步应聚焦的证据获取、搜索、推理或自我评估方向。为确保指导可靠性,采用GRPO训练专用评分生成器,其目标函数以多评委专家共识排名为基础,优化列表级斯皮尔曼相关性奖励,而非简单的成对偏好。在DeepResearchBench和SQA-CS-V2两个数据集上,无论使用8B/14B开源还是前沿闭源大模型构建的搜索代理,Co-ReAct均持续优于ReAct及代表性测试时计算优化基线。所训练的评分生成器可作为即插即用模块,无需修改原有决策机制即可提升性能。代码已公开于https://github.com/ZBWpro/Co-ReAct。
原文摘要 · Abstract (English)
ReAct-style agents for search-intensive, multi-step reasoning tasks rely largely on their own internal judgment to decide what evidence to seek, which reasoning or action step to take next, and when to stop, often producing shallow, redundant, or poorly targeted trajectories. Prior work has explored rubrics as external quality signals, but existing uses are mostly evaluative rather than action-guiding: rubrics typically serve as training-time rewards or post-hoc evaluators of completed outputs, and in deep-research settings they are often coarse-grained and report-level rather than step-level. We introduce Co-ReAct, a rubric-guided action-selection framework that uses rubrics as step-level guidance during inference. At each decision step, Co-ReAct injects a rubric into the agent's context to guide the next Reason-or-Act decision, specifying what the agent should target in evidence seeking, search, reasoning, or self-evaluation. To make this guidance reliable, we train a dedicated rubric generator with GRPO. Unlike prior pairwise or binary preference formulations, our objective optimizes a list-wise Spearman rank-correlation reward against multi-judge expert consensus rankings, encouraging rubrics that are discriminative rather than merely plausible. On DeepResearchBench and SQA-CS-V2, Co-ReAct consistently improves over ReAct and representative test-time compute baselines across search agents built on both 8B/14B open-source and frontier closed-source base models. The trained rubric generator can also serve as a drop-in component that improves these baselines without changing their underlying decision mechanisms. Our code is publicly available at https://github.com/ZBWpro/Co-ReAct.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。