让智能体通过经验学习，自动优化复杂问题的求解策略。
APEx: Distillation of Agent Procedural Experience for Adaptive Deep Research Question Answering
- 构建分层经验体系，将对话历史拆分为具体任务记忆和通用技能。
- 训练后技能可直接指导新任务，使模型在测试时自我改进14.7分。
- 适合需要持续学习与自适应推理的研究型AI系统开发者。
深度研究智能体通过多轮推理和外部工具回答复杂长周期问题。现有方法或依赖冗长的任务特定轨迹，影响决策效率；或提炼过程技能但与下游策略脱节。本文提出APEx，一种分层经验利用框架，将交互历史组织为实例级轨迹记忆和类别级过程技能，并通过执行器、蒸馏器和规划器组成的闭环架构耦合二者。三模块采用三阶段交替GRPO训练，实现奖励驱动的技能蒸馏而非固定提示生成。测试时，蒸馏技能作为过程先验，通过技能引导的测试时强化学习在线适配规划器，实现无真实答案的自我优化，并以技能对齐正则化防止策略漂移。在7个基准上实验显示，APEx性能达到当前最优，超越GPT-5.4达14.7分，较最强的记忆增强基线提升3.0分。
原文摘要 · Abstract (English)
Deep research agents augment large language models with external tools to answer complex, long-horizon questions through multi-turn reasoning. Learning from prior experience is crucial for continual improvement, yet existing methods either retrieve verbose task-specific traces that burden decision-making, or distill procedural skills that remain decoupled from downstream policy adaptation. We propose APEx, a hierarchical experience utilization framework that organizes interaction history into instance-level trajectory memories and category-level procedural skills, and couples them through a closed-loop architecture of Executor, Distiller, and Planner. The three modules are optimized via a three-stage alternating GRPO training paradigm, enabling reward-guided skill distillation rather than fixed-prompt generation. At test time, distilled skills serve as procedural priors for online Planner adaptation through skill-guided test-time reinforcement learning, allowing ground-truth-free self-improvement with skill-alignment regularization to prevent policy drift. Experiments on 7 benchmarks demonstrate that APEx achieves state-of-the-art performance, surpassing GPT-5.4 by 14.7 points and the strongest memory-augmented baseline by 3.0 points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。