用新基准和提示方法提升大模型在网格地图中的路径规划能力
GridRoute: A Benchmark for LLM-Based Route Planning with Cardinal Movement in Grid Environments
- 设计混合提示法AoT，让大模型结合传统算法指导路径推理
- 在多种地图规模下测试6个大模型，小模型提升最明显
- 适合研究大模型与经典算法融合的开发者和研究人员
大语言模型在规划与推理任务中展现出潜力，但多数研究仅关注其独立推理能力，忽视了与传统算法的协同可能。为此，我们提出全面评估基准GridRoute，用于衡量大模型如何利用传统算法。同时提出一种新型混合提示技术Algorithm of Thought（AoT），将传统算法引导融入提示过程。基准评估了从7B到72B参数的六种大模型，在不同尺寸的网格地图上测试其正确性、最优性和效率。结果表明，AoT显著提升各模型性能，尤其在更大或更复杂环境中表现突出，为解决路径规划挑战提供新思路。代码已开源：https://github.com/LinChance/GridRoute。
原文摘要 · Abstract (English)
Recent advancements in Large Language Models (LLMs) have demonstrated their potential in planning and reasoning tasks, offering a flexible alternative to classical pathfinding algorithms. However, most existing studies focus on LLMs' independent reasoning capabilities and overlook the potential synergy between LLMs and traditional algorithms. To fill this gap, we propose a comprehensive evaluation benchmark GridRoute to assess how LLMs can take advantage of traditional algorithms. We also propose a novel hybrid prompting technique called Algorithm of Thought (AoT), which introduces traditional algorithms' guidance into prompting. Our benchmark evaluates six LLMs ranging from 7B to 72B parameters across various map sizes, assessing their performance in correctness, optimality, and efficiency in grid environments with varying sizes. Our results show that AoT significantly boosts performance across all model sizes, particularly in larger or more complex environments, suggesting a promising approach to addressing path planning challenges. Our code is open-sourced at https://github.com/LinChance/GridRoute.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。