为深度搜索智能体设计了基于溯源的奖励分配机制，提升检索准确性。
STAMP: Provenance-Guided Credit Assignment for Deep Search Agents
- 通过文档溯源验证和首次曝光归因，精准追踪每条引用的支持来源。
- 在三个数据集上相比基线提升2.0~5.5分，且不改变原有轨迹排名。
- 适用于需要高可信度引用的智能搜索系统，尤其适合学术或严谨场景。
深度搜索智能体的强化学习主要关注轨迹级评分——如结果正确性、引用感知奖励与证据覆盖度。然而，暴露支持性文档的动作未获得针对性奖励，这被称为奖励-信用错配。本文提出STAMP：基于参考的验证器判断每篇引用文献是否支持训练时证据图中的实体或关系，并通过首次曝光归因将信用追溯至首次展示该文献的动作。该步骤信用通过保号优势调制注入，重新分配优势值但不改变轨迹级奖励或组内轨迹相对排名。在BrowseComp、BrowseComp-ZH和xbench-DS上，STAMP在相同SFT初始化、训练数据和搜索工具条件下，相较GRPO基线分别提升+2.0/+5.5/+3.0分，且可兼容仅结果奖励与引用评分双重奖励。组件消融实验确认，溯源信用信号与保号优势调制均对性能提升有贡献。
原文摘要 · Abstract (English)
Reinforcement learning for deep-search agents has largely focused on trajectory-level scoring -- outcome correctness, citation-aware rewards, and evidence coverage. Yet the actions that expose supporting documents receive no targeted credit, a gap we call the reward-credit mismatch. We propose STAMP, in which a reference-based verifier judges whether each cited document supports an entity or relation in a training-time evidence graph, and first-exposure attribution traces each supported citation back to the action that first surfaced it. This step credit is injected through sign-preserving advantage modulation, which redistributes advantage across steps without changing the trajectory-level reward or the relative ranking of trajectories within each group. On BrowseComp, BrowseComp-ZH, and xbench-DS, STAMP improves the GRPO baseline by +2.0/+5.5/+3.0 points under matched SFT initialization, training data, and search tools, and composes with both outcome-only and citation-rubric base rewards. Component ablations confirm that the provenance-based credit signal and the sign-preserving advantage modulation each contribute to the gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。