通过文档检索提升小模型生成规划语言的准确率
Documentation Retrieval Improves Planning Language Generation
- 用轻量级流程整合文档检索与代码生成
- 在BlocksWorld上计划正确率从0%提升至80%以上
- 适合想用小模型做形式化规划的研究者
某些强大的大语言模型在零样本情况下已能生成如PDDL这样的规划语言。然而，多数参数量低于500亿的开源模型因这类语言资源稀缺，表现接近零。我们通过一系列轻量级流程，将文档检索、模块化代码生成和错误修正相结合，显著提升了性能。使用Llama-4-Maverick等模型，最佳方案在常见BlocksWorld领域中将计划正确率从0%提升至80%以上。尽管语法错误大幅减少，但在更复杂的领域仍存在语义错误，暴露出当前模型推理能力的根本局限。
原文摘要 · Abstract (English)
Certain strong LLMs have shown promise for zero-shot formal planning by generating planning languages like PDDL. Yet, the performance of most open-source models under 50B parameters has been reported to be close to zero due to the low-resource nature of these languages. We significantly improve their performance via a series of lightweight pipelines that integrates documentation retrieval with modular code generation and error refinement. With models like Llama-4-Maverick, our best pipeline improves plan correctness from 0% to over 80% on the common BlocksWorld domain. However, while syntactic errors are substantially reduced, semantic errors persist in more challenging domains, revealing fundamental limitations in current models' reasoning capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。