用规划器反馈提升大模型的符号化规范可靠性。
Toward Secure and Reliable PDDL Formalization of Large Language Models with Planner-in-the-Loop Feedback
- 引入规划器闭环框架，通过诊断修复不可执行规范。
- 在多领域基准上实现规划成功率显著提升。
- 适合安全敏感场景下的大模型部署评估。
规划通常需要可执行且可验证的符号化规范。对于部署于自主或决策支持系统的大型语言模型，此类形式化失败可能导致不可验证的决策、执行失败或下游行为不安全。我们提出NL-PDDL-Bench，一个基于自然语言到PDDL规范构建的多领域基准，具备规划器验证的可执行性，并通过物体数量控制难度。我们进一步提出规划器闭环框架，利用验证器和规划器诊断，通过局部修改修复不可执行规范。在此基础上，我们开发了规划器驱动的优化方案：结合参数高效的低秩适配微调、离线规划器生成的偏好对进行直接偏好优化，以及推理时规划器闭环修复，训练过程无需在线规划器调用。我们还提供统一评估套件，涵盖可解析性、可求解性、规范相似性及与规划器参考的一致性。在代表性模型家族上的实验显示，规划成功率和计划层面一致性显著提升，且在难度扩展和跨领域变化下更具鲁棒性。结果表明，外部可验证的形式化对大模型在安全或安保敏感规划系统中的可靠部署具有重要价值。代码与数据见：https://github.com/ibasicplan/NL-PDDL-Bench
原文摘要 · Abstract (English)
Planning often requires symbolic specifications that are both executable and verifiable. For large language models deployed in autonomous or decision-support systems, failures in such formalization may lead to unverifiable decisions, execution failures, or unsafe downstream behavior. We present NL-PDDL-Bench, a multi-domain benchmark for natural-language-to-PDDL specification construction with planner-verified executability and controlled difficulty scaling by object count. We further propose a planner-in-the-loop framework that uses validator and planner diagnostics to revise non-executable specifications through localized edits. Building on this infrastructure, we develop a planner-grounded optimization recipe that combines parameter-efficient Low-Rank Adaptation supervised fine-tuning, offline planner-derived preference pairs for Direct Preference Optimization, and inference-time planner-in-the-loop repair, without requiring online planner calls during training. We also provide a unified evaluation suite for parseability, solvability, specification similarity, and outcome-aware plan-level consistency against planner references. Experiments on representative model families show substantial gains in planner success and plan-level agreement, with improved robustness under difficulty scaling and cross-domain variation. These results highlight the value of externally verifiable formalization for reliable deployment of LLMs in safety- or security-sensitive planning systems. Code and data are available at: https://github.com/ibasicplan/NL-PDDL-Bench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。