arXiv:2502.04728cs.AI2025-02中稿 · TMLR2025被引 15

用大模型测试时扩展生成精确的规划符号模型,无需训练即可超50%成功率。

Generating Symbolic World Models via Test-time Scaling of Large Language Models

  • 通过测试时采样与语义化精炼提升LLM生成PDDL的能力
  • 在两个任务上达超50%成功率,优于o1-mini
  • 适合需要高精度规划的复杂任务场景

解决复杂规划问题需要大语言模型显式建模状态转移,以避免规则违反、满足约束并确保最优性,但自然语言固有的模糊性阻碍了这一过程。为此,本文采用规划领域定义语言(PDDL)作为规划抽象,实现精确且形式化的状态描述。借助PDDL,可构建符号世界模型,使A*等经典搜索算法无缝应用于最优路径求解。然而,当前大模型直接生成高质量PDDL领域仍面临挑战,主要因缺乏标注的PDDL训练数据。为应对该问题,本文提出利用大模型测试时计算扩展来增强其PDDL推理能力,从而生成高质量的PDDL领域。具体而言,设计了一种简单有效的算法:先使用Best-of-N采样提高初始解质量,再通过语义化机器学习进行细粒度优化。该方法在无额外训练条件下,于两项任务中均实现超过50%的成功率(即从自然语言描述或PDDL问题生成对应领域),显著优于o1-mini。由于采用PDDL作为状态抽象,本方法在几乎所有竞赛级规划任务中表现超越现有最先进水平。

原文摘要 · Abstract (English)

Solving complex planning problems requires Large Language Models (LLMs) to explicitly model the state transition to avoid rule violations, comply with constraints, and ensure optimality-a task hindered by the inherent ambiguity of natural language. To overcome such ambiguity, Planning Domain Definition Language (PDDL) is leveraged as a planning abstraction that enables precise and formal state descriptions. With PDDL, we can generate a symbolic world model where classic searching algorithms, such as A*, can be seamlessly applied to find optimal plans. However, directly generating PDDL domains with current LLMs remains an open challenge due to the lack of PDDL training data. To address this challenge, we propose to scale up the test-time computation of LLMs to enhance their PDDL reasoning capabilities, thereby enabling the generation of high-quality PDDL domains. Specifically, we introduce a simple yet effective algorithm, which first employs a Best-of-N sampling approach to improve the quality of the initial solution and then refines the solution in a fine-grained manner with verbalized machine learning. Our method outperforms o1-mini by a considerable margin in the generation of PDDL domains, achieving over 50\% success rate on two tasks (i.e., generating PDDL domains from natural language description or PDDL problems). This is done without requiring additional training. By taking advantage of PDDL as state abstraction, our method is able to outperform current state-of-the-art methods on almost all competition-level planning tasks.

规划生成PDDL大模型推理符号建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。