首个能在开放环境自主创建工具的遥感智能体,突破传统工具调用局限。
OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observation
- 通过自适应规划与动态创建工具,实现对未知数据和任务的泛化能力。
- 在跨基准测试中仅用6个预训练模型,性能媲美依赖104个专用工具的旧方法。
- 适合需要跨领域、自适应处理多源遥感数据的研究者与开发者。
地球观测(EO)对于感知地表动态变化至关重要,但在开放环境中实现自主地球观测仍面临多源数据多样性与异构任务的挑战。现有遥感智能体局限于封闭环境,依赖预定义工具且适用范围狭窄。为此,我们提出OpenEarth-Agent——首个面向开放环境地球观测的工具创建智能体框架。该框架不依赖预设工具,而是通过自适应工作流规划与工具生成,实现对未见数据与任务的泛化能力。其适应性依托于多阶段工具与跨领域知识库的开放式集成,支持在多个应用领域内完成完整的地球观测流程。为全面评估开放环境中的智能体表现,我们构建了OpenEarth-Bench,一个包含596个真实世界全链条案例的基准,覆盖七大应用领域,仅提供6个基础预训练模型工具,无任何特定任务工具。大量实验表明,OpenEarth-Agent可在开放环境中成功完成多领域全链条地球观测任务。值得注意的是,在跨基准测试Earth-Bench上,仅使用6个预训练模型的工具创建智能体,性能达到依赖104个专用工具的工具调用智能体水平,并在完整工具集下显著超越后者;部分自动生成工具对数据异常表现出优于人工设计工具的鲁棒性。
原文摘要 · Abstract (English)
Earth Observation (EO) is essential for perceiving dynamic land surface changes, yet deploying autonomous EO in open environments is hindered by the immense diversity of multi-source data and heterogeneous tasks. While remote sensing agents have emerged to streamline EO workflows, existing tool-calling agents are confined to closed environments. They rely on pre-defined tools and are restricted to narrow scope, limiting their generalization to the diverse data and tasks. To overcome these limitations, we introduce OpenEarth-Agent, the first tool-creation agent framework tailored for open-environment EO. Rather than calling predefined tools, OpenEarth-Agent employs adaptive workflow planning and tool creation to generalize to unseen data and tasks. This adaptability is bolstered by an open-ended integration of multi-stage tools and cross-domain knowledge bases, enabling robust execution in the entire EO pipeline across multiple application domains. To comprehensively evaluate EO agents in open environments, we propose OpenEarth-Bench, a novel benchmark comprising 596 real-world, full-pipeline cases across seven application domains, explicitly designed to assess agents' adaptive planning and tool creation capabilities. Only essential pre-trained model tools are provided in this benchmark, devoid of any other predefined task-specific tools. Extensive experiments demonstrate that OpenEarth-Agent successfully masters full-pipeline EO across multiple domains in the open environment. Notably, on the cross-benchmark Earth-Bench, our tool-creating agent equipped with 6 essential pre-trained models achieves performance comparable to tool-calling agents relying on 104 specialized tools, and significantly outperforms them when provided with the complete toolset. In several cases, the created tools exhibit superior robustness to data anomalies compared to human-engineered counterparts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。