arXiv:2605.16142cs.AIcs.LG2026-05被引 1

用属性验证替代评分,让大模型更高效生成高质量规划程序。

Property-Guided LLM Program Synthesis for Planning

论文配图:Property-Guided LLM Program Synthesis for Planning
图 1 · 摘自论文原文
  • 通过形式化属性验证代替数值评分,提前发现程序错误
  • 平均减少7倍程序生成量,计算成本降低数个数量级
  • 适合需高可靠性且可定义验证属性的程序合成任务

大模型在程序合成中表现优异,但依赖简单数值评分(如测试通过数)来判断质量,缺乏失败原因反馈,导致需大量试错,增加推理与评估成本。本文提出属性引导的程序合成方法:不等程序执行后才打分,而是检查候选程序是否满足形式化属性。一旦属性被违反,立即停止评估,并提供具体反例说明失败点,从而大幅减少生成次数与评估开销,有效引导大模型生成更强程序。在PDDL规划领域,要求合成直接启发式函数——所有严格改进状态均有严格改进后继。通过反例引导的修复循环,在训练集上检查属性并返回首个违反项。在十个规划领域上测试,新方法在几乎所有测试任务上均实现有效直接性;相比最优基线,平均每个领域少生成7倍程序,无需搜索即可解决更多任务,候选评估计算量减少数个数量级。只要问题具有可验证属性,该方法就能显著降低成本并提升程序质量。

原文摘要 · Abstract (English)

LLMs have shown impressive success in program synthesis, discovering programs that surpass prior solutions. However, these approaches rely on simple numeric scores to signal program quality, such as the value of the solution or the number of passed tests. Because a score offers no guidance on why a program failed, the system must generate and evaluate many candidates hoping some succeed, increasing LLM inference and evaluation costs. We study a different approach: property-guided LLM program synthesis. Instead of scoring programs after evaluation, we check whether a candidate satisfies a formally defined property. When the property is violated, we stop the evaluation early and provide the LLM with a concrete counterexample showing exactly how the program failed. This feedback drastically reduces both the number of program generations and the evaluation cost, and can guide the LLM to generate stronger programs. We evaluate this approach on PDDL planning domains, asking the LLM to synthesize direct heuristic functions: every state reachable by strictly improving transitions has a strictly improving successor. A heuristic with this property leads hill-climbing algorithm directly to a goal state. A counterexample-guided repair loop generates one candidate program, checks the property over a training set, and returns the first case that violates the property. We evaluate our approach on ten planning domains with an out-of-distribution test set. The synthesized heuristics are effectively direct on virtually all test tasks, and compared to the best prior generation method our approach generates seven times fewer programs per domain on average, solves more tasks without using search, and requires several orders of magnitude less computation to evaluate candidates. Whenever a problem admits a verifiable property, property-guided LLM synthesis can reduce cost and improve program quality.

程序合成属性验证规划大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。