arXiv:2608.01359cs.CL2026-08

用证据引导的自蒸馏,让搜索型语言代理更精准地学习每一步操作。

EviSD: Evidence-Conditioned Self-Distillation for Search-Augmented Agents

论文配图:EviSD: Evidence-Conditioned Self-Distillation for Search-Augmented Agents
图 1 · 摘自论文原文
  • 用支持证据作为动作指引,黄金答案辅助答案决策。
  • 在7个问答基准上,准确率最高提升2.3点,仅修改6.7%~15.1%的输出。
  • 无需额外目标或推理改动,适合大规模语言模型优化。

基于结果的强化学习使搜索增强型语言代理能从可验证的最终答案中学习,但其轨迹级信用分配无法区分多轮搜索中各动作的贡献。我们提出EviSD,一种证据条件自蒸馏框架,利用实例级支持证据作为搜索动作的特权信息,黄金答案作为答案动作的补充特权。训练时,学生从原始上下文采样动作,同一模型在对齐上下文中重新评分这些动作作为特权教师。EviSD将师生差距转化为对结果驱动的GRPO优势的有界修正,并仅应用于生成的动作片段。该设计在保留结果奖励决定的更新方向的同时,局部化了特权引导,无需辅助蒸馏目标,且推理阶段无变化。在七个问答基准和三种不同规模、代际的骨干模型上,EviSD在所有评估设置中均取得最高宏平均精确匹配率,优于最强对比方法1.3–2.3点,仅调节6.7%–15.1%的响应标记。代码见https://github.com/JiananXie/EviSD。

原文摘要 · Abstract (English)

Outcome-based reinforcement learning enables search-augmented language agents to learn from verifiable final answers, but its trajectory-level credit cannot distinguish the contributions of individual actions in a multi-turn search process. We propose EviSD, an evidence-conditioned self-distillation framework that uses instance-level supporting evidence as privileged information for search actions and golden answers as complementary privilege for answer actions. During training, the student samples actions from the original context, while the same model re-scores them as a privileged teacher under an action-aligned context. EviSD converts the detached teacher--student gap into a bounded correction to the outcome-derived GRPO advantage and applies it only to generated action spans. This design localizes privileged guidance while preserving the update direction determined by the outcome reward, without an auxiliary distillation objective or any change at inference time. Across seven question-answering benchmarks and three backbones spanning model scales and generations, EviSD achieves the highest macro-average Exact Match in all evaluated settings, outperforming the strongest compared methods by 1.3--2.3 points while modulating only 6.7%--15.1% of response tokens. Code is available at https://github.com/JiananXie/EviSD.

强化学习自蒸馏问答系统搜索代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。