用策略抽象提升大模型长程决策能力，让智能体更会规划。
StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction
- 从任务初始状态采样策略，约束后续动作，形成分层决策机制
- 在ALFWorld和WebShop上分别达到93.1%和84.2%的成功率
- 适合需要长期规划与高效探索的复杂交互任务研究
大型语言模型日益作为交互智能体使用，但其在长时决策优化中仍面临挑战，因现有方法多为纯反应式，削弱了长期轨迹上的探索与信用分配。本文提出战略轨迹抽象（StraTA），一种在代理强化学习中引入显式轨迹级策略的简单框架。StraTA从初始任务状态采样紧凑策略，以该策略条件化后续动作，并通过分层GRPO风格的滚动设计，联合训练策略生成与动作执行，进一步结合多样化策略滚动与关键性自我判断机制。在ALFWorld、WebShop和SciWorld上的实验表明，StraTA在样本效率和最终性能上均优于强基线。在ALFWorld上取得93.1%的成功率，在WebShop上达84.2%；在SciWorld上获得63.5%的整体得分，超越前沿闭源模型。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used as interactive agents, but optimizing them for long-horizon decision making remains difficult because current methods are largely purely reactive, which weakens both exploration and credit assignment over extended trajectories. In this work, we present Strategic Trajectory Abstraction (StraTA), a simple framework that introduces an explicit trajectory-level strategy into agentic reinforcement learning (RL). StraTA samples a compact strategy from the initial task state, conditions subsequent actions on that strategy, and trains strategy generation and action execution jointly with a hierarchical GRPO-style rollout design, further enhanced by diverse strategy rollout and critical self-judgment. Experiments on ALFWorld, WebShop, and SciWorld show that StraTA consistently improves both sample efficiency and final performance over strong baselines. StraTA reaches success rates of 93.1% on ALFWorld and 84.2% on WebShop. On SciWorld, StraTA attains a 63.5% overall score, outperforming frontier closed-source models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。