测试大模型能否证明机器人路径规划算法的最优性,首次构建研究级验证基准。
Can LLMs Prove Robotic Path Planning Optimality? A Benchmark for Research-Level Algorithm Verification
- 构建34个路径规划优化证明任务,需结合算法描述与复杂约束进行多步推理。
- 顶尖大模型在无外部知识时仅能生成部分有效证明,准确率不足。
- 提供特定领域引理可显著提升推理质量,优于通用思维链提示。
机器人路径规划问题通常为NP难,实际解法依赖具有理论保证的近似算法。设计此类算法已具挑战,而形式化证明其近似最优性更需领域几何洞察与复杂约束下的多步数学推演。尽管大语言模型(LLMs)在数学推理任务中表现优异,但其在机器人路径规划研究级最优性证明中的能力仍待探索。本文提出首个用于评估LLMs在路径规划近似比证明方面能力的基准,包含34个涵盖不同问题类型与复杂度的研究级证明任务,每项均需对算法描述、问题约束和理论保证进行结构化推理。对主流专有及开源LLMs的评估显示,即使最强模型在缺乏领域知识时也难以生成完整有效证明。然而,通过提供任务相关的上下文引理,可显著提升推理质量,该策略效果优于通用思维链提示或事后提供真实近似比。我们进一步开展细粒度错误分析,揭示常见逻辑谬误与幻觉类型,并展示如何通过针对性上下文增强加以缓解。
原文摘要 · Abstract (English)
Robotic path planning problems are often NP-hard, and practical solutions typically rely on approximation algorithms with provable performance guarantees for general cases. While designing such algorithms is challenging, formally proving their approximation optimality is even more demanding, which requires domain-specific geometric insights and multi-step mathematical reasoning over complex operational constraints. Recent Large Language Models (LLMs) have demonstrated strong performance on mathematical reasoning benchmarks, yet their ability to assist with research-level optimality proofs in robotic path planning remains under-explored. In this work, we introduce the first benchmark for evaluating LLMs on approximation-ratio proofs of robotic path planning algorithms. The benchmark consists of 34 research-grade proof tasks spanning diverse planning problem types and complexity levels, each requiring structured reasoning over algorithm descriptions, problem constraints, and theoretical guarantees. Our evaluation of state-of-the-art proprietary and open-source LLMs reveals that even the strongest models struggle to produce fully valid proofs without external domain knowledge. However, providing LLMs with task-specific in-context lemmas substantially improves reasoning quality, a factor that is more effective than generic chain-of-thought prompting or supplying the ground-truth approximation ratio as posterior knowledge. We further provide fine-grained error analysis to characterize common logical failures and hallucinations, and demonstrate how each error type can be mitigated through targeted context augmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。