让智能体在检索后立即自评，提升复杂问答的准确性
Evaluate-as-Action: Self-Evaluated Process Rewards for Retrieval-Augmented Agents
- 检索后强制执行结构化自评，生成过程奖励信号
- 多跳任务准确率显著提升，平均表现优于现有方法
- 适合需要精细推理的开放域问答系统研究者
检索增强型智能体可查询外部证据，但其在多步推理中的可靠性仍受限：噪声检索可能导致多跳问答失败，而仅基于结果的强化学习提供的信用信号过于粗粒度，难以优化中间步骤。我们提出 extsc{EvalAct}（Evaluate-as-Action），将隐式的检索质量评估转化为显式动作，并强制执行搜索-评估协同协议，使每次检索后立即生成结构化评估分数，从而获得与交互轨迹对齐的过程信号。为利用这些信号，我们引入基于 GRPO 的过程校准优势重标定（PCAR）方法，根据评估分数在段级重标定优势，强调可靠段落，对不确定段落保守更新。在七个开放域问答基准上的实验表明， extsc{EvalAct} 实现了最佳平均准确率，尤其在多跳任务上增益最大；消融实验验证了显式评估循环是主要提升来源，而 PCAR 提供持续附加收益。
原文摘要 · Abstract (English)
Retrieval-augmented agents can query external evidence, yet their reliability in multi-step reasoning remains limited: noisy retrieval may derail multi-hop question answering, while outcome-only reinforcement learning provides credit signals that are too coarse to optimize intermediate steps. We propose \textsc{EvalAct} (Evaluate-as-Action), which converts implicit retrieval quality assessment into an explicit action and enforces a coupled Search-to-Evaluate protocol so that each retrieval is immediately followed by a structured evaluation score, yielding process signals aligned with the interaction trajectory. To leverage these signals, we introduce Process-Calibrated Advantage Rescaling (PCAR), a GRPO-based optimization method that rescales advantages at the segment level according to evaluation scores, emphasizing reliable segments while updating uncertain ones conservatively. Experiments on seven open-domain QA benchmarks show that \textsc{EvalAct} achieves the best average accuracy, with the largest gains on multi-hop tasks, and ablations verify that the explicit evaluation loop drives the primary improvements while PCAR provides consistent additional benefits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。