arXiv:2608.17341cs.AI2026-08

用大模型自动修复规划模型错误,效果显著但不靠谱。

LLM-Only PDDL Domain Repair with Open-Weight Models

  • 直接用大模型修改PDDL规划模型,无需额外符号工具
  • 最佳模型F1达0.87,但测试通过率仅0.92,关键域低至0.06
  • 适合研究大模型在符号推理中的边界,不适合高可靠场景

AI规划旨在寻找达成目标的动作序列,依赖显式的世界模型,通常以规划领域定义语言(PDDL)表示。当前研究聚焦于检测并修复此类模型中的错误。例如,用户可提供成功执行的正例计划和失败的负例计划,自动化修复方法据此修改PDDL模型以满足约束。本文评估近期开源大语言模型在纯大模型架构下执行此修复任务的能力。实验显示,符号基线F1为0.49,最佳大模型达0.87,绝对提升0.38,但平均测试通过率仅为0.82,在Thoughtful领域降至0.06;即使包含测试轨迹的最佳设置也仅达0.92。因此,当前开源大模型无法保证满足可靠自动化模型修复所需的测试约束。

原文摘要 · Abstract (English)

AI planning is concerned with finding a sequence of actions that achieves a specified goal. It relies on explicit models of the world, commonly represented in the Planning Domain Definition Language (PDDL). An active line of research investigates how errors in such models can be detected and repaired. For example, users may provide positive test plans that are solutions, and negative test plans that fail during execution. Automated repair methods then modify the PDDL model to satisfy these constraints. In this paper, we evaluate the ability of recent open-weight large language models to perform this repair task using an LLM-only approach. Our experiments show that the symbolic baseline achieves an $F_1$ score of $.49$, while the best-performing LLM reaches $.87$ with high reasoning effort, an absolute improvement of $.38$. However, that setting has a mean test pass rate of only $.82$, falling to $.06$ on the Thoughtful domain; even the best setting that includes the test traces reaches only $.92$. Thus, current open-weight models cannot guarantee satisfaction of the test constraints required for reliable automated model repair.

AI规划大模型PDDL修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。