用大模型生成动作序列，减少与环境交互次数。
Query-Efficient Planning with Language Models
- 让大模型直接生成完整行动序列，而非仅辅助搜索。
- 相比基线方法，交互次数减少50%以上，效率显著提升。
- 适合需要快速适应反馈的智能体规划任务。
在复杂环境中进行规划，要求智能体高效查询世界模型，以找到从起点到目标的可行动作序列。近期研究显示，大型语言模型（LLMs）凭借其丰富的先验知识和推理能力，可通过在有前景的状态空间中搜索并根据环境反馈调整策略，助力规划。本文提出并研究两种根本竞争的框架：第一种将LLM作为启发式方法嵌入搜索规划器，用于选择待拓展节点并提出有前景的动作；第二种则将LLM作为生成式规划器，从起点直接生成完整动作序列，经由世界模型验证后根据反馈进行调整。实验表明，尽管两种方法均优于基线，但使用LLM作为生成式规划器可显著减少与环境的交互次数。关键发现是：相较于作为启发式的LLM，作为规划器的LLM能更快地根据即时反馈调整策略。我们在Robotouille和PDDL规划基准上进行了评估与消融实验，并讨论了与现有查询效率规划算法理论的关联。代码已公开于https://github.com/portal-cornell/llms-for-planning。
原文摘要 · Abstract (English)
Planning in complex environments requires an agent to efficiently query a world model to find a feasible sequence of actions from start to goal. Recent work has shown that Large Language Models (LLMs), with their rich prior knowledge and reasoning capabilities, can potentially help with planning by searching over promising states and adapting to feedback from the world. In this paper, we propose and study two fundamentally competing frameworks that leverage LLMs for query-efficient planning. The first uses LLMs as a heuristic within a search-based planner to select promising nodes to expand and propose promising actions. The second uses LLMs as a generative planner to propose an entire sequence of actions from start to goal, query a world model, and adapt based on feedback. We show that while both approaches improve upon comparable baselines, using an LLM as a generative planner results in significantly fewer interactions. Our key finding is that the LLM as a planner can more rapidly adapt its planning strategies based on immediate feedback than LLM as a heuristic. We present evaluations and ablations on Robotouille and PDDL planning benchmarks and discuss connections to existing theory on query-efficient planning algorithms. Code is available at https://github.com/portal-cornell/llms-for-planning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。