验证大模型计划生成效果，发现其不如传统贪心搜索。
A complementary study on PlanGPT: Evaluation with defined Performance Metrics and comparison with a planner
- 用计划成本和生成时间评估PlanGPT性能
- 结果表明其表现不及贪心搜索策略
- 适合关注LLM在规划中实际效用的研究者
自动规划是人工智能的一个子领域，目标是从初始状态生成一系列动作（即计划）以达到目标状态。规划问题由对象、初始状态和目标状态定义，旨在计算从初始状态到目标状态的可行路径。生成计划的程序称为规划器。本文对去年发布的先进大模型PlanGPT进行了补充研究，重新实验以验证使用大模型进行规划是否具有实际意义。我们检查了原论文中关于计划覆盖率的结果是否准确，并对PlanGPT的性能进行了更全面的评估，采用计划成本（Plan Cost）和计划生成时间（Plan Generation Time）两个指标，与传统规划器在相同任务上的表现进行对比。结果显示，PlanGPT的表现并不优于贪心搜索策略。
原文摘要 · Abstract (English)
Automated Planning is a subfield of Artificial Intelligence (AI) where the main objective is generating a sequence of actions, known as a plan, that helps us reach a goal state from an initial state. A planning problem is defined by a set of objects, an initial state and a desired goal state. The objective is to compute a plan that'll lead us from the inital state to the goal state. Programs that generate plans are called planners. In this paper, we did a complementary study to the state-of-the-art LLM called PlanGPT which was released last year. We redid some experiments to verify whether planning with LLMs is \textbf{pertinent} and \textbf{worthwhile}. We also check whether the results obtained in the official PlanGPT paper for plan coverage were correct, and we also performed a more comprehensive study on PlanGPT's performance: in our paper PlanGPT's performance was evaluated using two metrics: Plan Cost and Plan Generation Time. The results of planGPT were compared to those produced by a traditional planner for the same plans and same metrics. We discovered that PlanGPT is no better than a Greedy search strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。