arXiv:2601.10029cs.AI2026-01被引 6

PaperScout用动态决策代替固定流程,让论文搜索更智能高效。

PaperScout: An Autonomous Agent for Academic Paper Search with Process-Aware Sequence-Level Policy Optimization

  • 将论文搜索建模为连续决策过程,根据上下文动态调用工具。
  • 在真实和合成数据集上,召回率与相关性均显著优于基线方法。
  • 提出序列级优化算法PSPO,解决多轮交互中的训练不稳问题。

学术论文搜索是科研的基础任务,但现有方法多依赖僵化的预设流程,难以应对复杂条件查询。为此,我们提出PaperScout,一种将论文搜索重构为序贯决策过程的自主代理。与静态工作流不同,PaperScout能根据累积检索上下文,动态决定何时、是否及如何调用搜索与扩展工具。然而,训练此类代理面临根本挑战:标准强化学习方法通常针对单轮任务设计,在多轮代理任务中存在粒度错配——令牌级优化与序列级交互不一致,导致信用分配噪声大、训练不稳定。我们提出近端序列策略优化(PSPO),一种面向流程、基于序列级别的策略优化方法,使优化对齐于代理-环境交互。在合成与真实世界基准上的全面实验表明,PaperScout在召回率与相关性上显著优于强基线的工作流驱动与强化学习方法,验证了自适应代理框架与优化策略的有效性。

原文摘要 · Abstract (English)

Academic paper search is a fundamental task in scientific research, yet most existing approaches rely on rigid, predefined workflows that struggle with complex, conditional queries. To address this limitation, we propose PaperScout, an autonomous agent that reformulates paper search as a sequential decision-making process. Unlike static workflows, PaperScout dynamically decides whether, when, and how to invoke search and expand tools based on accumulated retrieval context. However, training such agents presents a fundamental challenge: standard reinforcement learning methods, typically designed for single-turn tasks, suffer from a granularity mismatch when applied to multi-turn agentic tasks-where token-level optimization diverges from the granularity of sequence-level interactions-leading to noisy credit assignment and unstable training dynamics. We introduce Proximal Sequence Policy Optimization (PSPO), a process-aware, sequence-level policy optimization method that aligns optimization with agent--environment interaction. Comprehensive experiments on both synthetic and real-world benchmarks demonstrate that PaperScout significantly outperforms strong workflow-driven and RL baselines in both recall and relevance, validating the effectiveness of our adaptive agentic framework and optimization strategy.

自主代理论文搜索强化学习序列优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。