让大模型计划与执行对齐,通过迭代优化路径提升成功率。
PIVOT: Bridging Planning and Execution in LLM Agents via Trajectory Refinement

- 将计划路径视为可优化对象,通过环境反馈逐步改进。
- 在DeepPlanning和GAIA上实现约束满足率最高94%的提升。
- 无需人工监督也能有效运行,效率比同类方法高3到5倍。
基于大语言模型(LLM)的智能体常生成看似合理但执行失败的计划,原因包括不可行动作、违反约束及长期累积误差。PIVOT(Plan-Inspect-eVOlve Trajectories)提出一种自监督框架,将轨迹视为可优化对象,通过环境交互迭代优化。该框架包含四个阶段:PLAN生成候选轨迹;INSPECT执行并计算结构化损失,文本梯度编码计划与执行的差异;EVOLVE利用这些信号生成改进轨迹;VERIFY进行最终全局约束检查。单调接受机制确保解质量不下降。在DeepPlanning和GAIA上的实证表明,结合人工反馈(HITL)时,约束满足率相对提升最高达94%;全自主版本仍保持显著优势,证明核心轨迹优化机制无需外部监督依然有效。同时,PIVOT计算高效,所需令牌数仅为竞品方法的1/3至1/5。结果表明,基于反馈的轨迹优化是弥合自主智能体计划-执行差距的可靠方法。
原文摘要 · Abstract (English)
Large language model (LLM)-based agents frequently generate seemingly coherent plans that fail upon execution due to infeasible actions, constraint violations, and compounding errors over extended horizons. PIVOT (Plan-Inspect-eVOlve Trajectories) addresses this plan-execution misalignment through a self-supervised framework that treats trajectories as optimizable objects iteratively refined via environment interaction. The framework comprises four stages: PLAN generates candidate trajectories; INSPECT executes them and computes structured losses with textual gradients encoding plan-execution discrepancies; EVOLVE applies these signals to produce improved trajectories; and VERIFY performs a final global check against task constraints. A monotonic acceptance process ensures a non-decreasing solution quality. Empirical evaluations on DeepPlanning and GAIA demonstrate state-of-the-art performance: with human-in-the-loop (HITL) feedback, PIVOT establishes a strong upper bound up to 94% relative improvement in constraint satisfaction, while its fully autonomous variant retains substantial gains, showing that the core trajectory-refinement mechanism remains effective without external supervision. At the same time, PIVOT remains computationally efficient, requiring up to 3x to 5x fewer tokens than competing refinement methods. These findings establish that (self- or human-supervised) feedback-based trajectory optimization is a principled methodology for mitigating plan-execution gaps in autonomous agent systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。