arXiv:2410.18792cs.CYcs.CL2024-10被引 31

提出GeoAgent框架,让大模型更准地处理地理空间数据任务。

An LLM Agent for Automatic Geospatial Data Analysis

  • 用MCTS结合代码解释器与RAG,实现多步推理
  • 在复杂地理任务中函数调用成功率提升42%
  • 适合想自动化地理分析的研究者和开发者

大语言模型在数据科学代码生成中常因复杂序列任务而出现逻辑错误,尤其在地理空间数据处理中面临数据结构复杂、空间约束难融入、函数调用多样及对冷门库幻觉等问题。为此,我们提出GeoAgent,一个基于蒙特卡洛树搜索(MCTS)的交互式框架,首次集成代码解释器、静态分析与检索增强生成(RAG)技术,以提升地理空间数据处理能力。同时,我们构建了一个新基准,涵盖数据获取、分析与可视化等单轮与多轮任务,使用多种Python库评估不同场景。结果表明,仅依赖模型知识无法准确完成地理任务,需连贯的多步流程与多次函数调用。相比基线模型,GeoAgent在函数调用成功率和任务完成率上均有显著提升,验证了其有效性,为未来自动地理分析工具开发提供重要参考。

原文摘要 · Abstract (English)

Large language models (LLMs) are being used in data science code generation tasks, but they often struggle with complex sequential tasks, leading to logical errors. Their application to geospatial data processing is particularly challenging due to difficulties in incorporating complex data structures and spatial constraints, effectively utilizing diverse function calls, and the tendency to hallucinate less-used geospatial libraries. To tackle these problems, we introduce GeoAgent, a new interactive framework designed to help LLMs handle geospatial data processing more effectively. GeoAgent pioneers the integration of a code interpreter, static analysis, and Retrieval-Augmented Generation (RAG) techniques within a Monte Carlo Tree Search (MCTS) algorithm, offering a novel approach to geospatial data processing. In addition, we contribute a new benchmark specifically designed to evaluate the LLM-based approach in geospatial tasks. This benchmark leverages a variety of Python libraries and includes both single-turn and multi-turn tasks such as data acquisition, data analysis, and visualization. By offering a comprehensive evaluation among diverse geospatial contexts, this benchmark sets a new standard for developing LLM-based approaches in geospatial data analysis tasks. Our findings suggest that relying solely on knowledge of LLM is insufficient for accurate geospatial task programming, which requires coherent multi-step processes and multiple function calls. Compared to the baseline LLMs, the proposed GeoAgent has demonstrated superior performance, yielding notable improvements in function calls and task completion. In addition, these results offer valuable insights for the future development of LLM agents in automatic geospatial data analysis task programming.

大模型地理分析代码生成智能代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。