用分步模拟让大模型当智能规划员,效果比直接调用更好。
Agentic LLM Planning via Step-Wise PDDL Simulation: An Empirical Characterisation
- 大模型逐个选动作,每步后观察状态并可重试,不一次性定全部计划。
- 在102个测试任务中,交互式规划成功率66.7%,比直接调用高3个百分点。
- 适合研究大模型规划能力的学者,尤其关注反馈机制如何影响表现。
任务规划是自主机器人系统的核心能力,即从初始状态通过一系列动作达到目标。大型语言模型(LLM)能否作为经典符号方法的可行替代方案仍存疑问。本文提出PyPDDLEngine,一个开源的规划领域定义语言(PDDL)仿真引擎,通过模型上下文协议(MCP)接口将规划操作暴露为LLM工具调用。与一次性确定完整动作序列不同,该方法让LLM充当交互式搜索策略:每次选择一个动作,观察执行后的状态,并可重置和重试。我们在统一180秒预算下,在102个国际规划竞赛(IPC)Blocksworld实例上评估四种方法:经典基线Fast Downward lama-first和seq-sat-lama-2011,直接调用大模型(Claude Haiku 4.5),以及通过PyPDDLEngine实现的代理式大模型规划。结果显示,Fast Downward成功率为85.3%;直接与代理式大模型分别为63.7%和66.7%,代理方式仅提升3个百分点,但代价高达5.7倍。在多数共解难度区块中,两种大模型方法生成的计划长度均短于seq-sat-lama-2011,这可能源于训练数据回忆而非可泛化规划。结果表明,代理式收益依赖环境反馈性质:编程代理受益于编译错误等外部信号,而PDDL步骤反馈为自评,缺乏外部验证,导致自我评估局限。
原文摘要 · Abstract (English)
Task planning, the problem of sequencing actions to reach a goal from an initial state, is a core capability requirement for autonomous robotic systems. Whether large language models (LLMs) can serve as viable planners alongside classical symbolic methods remains an open question. We present PyPDDLEngine, an open-source Planning Domain Definition Language (PDDL) simulation engine that exposes planning operations as LLM tool calls through a Model Context Protocol (MCP) interface. Rather than committing to a complete action sequence upfront, the LLM acts as an interactive search policy that selects one action at a time, observes each resulting state, and can reset and retry. We evaluate four approaches on 102 International Planning Competition (IPC) Blocksworld instances under a uniform 180-second budget: Fast Downward lama-first and seq-sat-lama-2011 as classical baselines, direct LLM planning (Claude Haiku 4.5), and agentic LLM planning via PyPDDLEngine. Fast Downward achieves 85.3% success. The direct and agentic LLM approaches achieve 63.7% and 66.7%, respectively, a consistent but modest three-percentage-point advantage for the agentic approach at $5.7\times$ higher token cost per solution. Across most co-solved difficulty blocks, both LLM approaches produce shorter plans than seq-sat-lama-2011 despite its iterative quality improvement, a result consistent with training-data recall rather than generalisable planning. These results suggest that agentic gains depend on the nature of environmental feedback. Coding agents benefit from externally grounded signals such as compiler errors and test failures, whereas PDDL step feedback is self-assessed, leaving the agent to evaluate its own progress without external verification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。