arXiv:2605.12678cs.CVcs.CY2026-05被引 7

地理空间大模型缺乏统一评估标准,导致无法判断哪个最先进。

No One Knows the State of the Art in Geospatial Foundation Models

  • 通过审计152篇论文发现评估标准混乱
  • 超九成论文使用唯一预训练配置,无法复现
  • 呼吁建立共享评估框架与权重公开规范

地理空间基础模型(GFMs)被提出作为灾害响应、土地覆盖制图、粮食安全监测等高风险地球观测任务的通用骨干。然而,现有文献未能提供足够信息以判断哪种模型适用于特定任务。我们指出:当前无人真正知晓地理空间基础模型的最先进技术状态。方法虽有潜力,但该领域在评估、训练测试协议、权重发布及预训练控制方面缺乏标准化,致使模型无法比较或排序。在对152篇论文的审计中,我们发现46次跨论文差异超过10分(同一模型、基准、协议下);126篇可提取预训练数据的论文中,94篇采用其他论文未使用的配置;39%的论文未发布模型权重。这一差距源于协作失败,而非个别实验室过错。本文提出六项具体期望:命名许可的权重发布、共享核心评估、复制对比与重跑基线标注、方差报告、单一共享评估工具包,以及数据-架构-算法的控制。我们旨在推动建立对地理空间基础模型创新的共同理解。

原文摘要 · Abstract (English)

Geospatial foundation models (GFMs) have been proposed as generalizable backbones for disaster response, land-cover mapping, food-security monitoring, and other high-stakes Earth-observation tasks. Yet the published work about these models does not give reviewers or users enough information to tell which model fits a given task. We argue that nobody knows what the current state of the art is in geospatial foundation models. The methods may be useful, but the GFM literature does not standardize evaluations, training and testing protocols, released weights, or pretraining controls well enough for anyone to compare or rank them. In a 152-paper audit, we find 46 cross-paper disagreements of at least 10 points for the same model, benchmark, and protocol; 94/126 papers with extractable pretraining data use a configuration no other paper uses; and 39% of GFM papers release no model weights. This lack of community standards can be solved. We propose six concrete expectations: named-license weight release, shared core evaluations, copied-versus-rerun baseline annotations, variance reporting, one shared evaluation harness, and data-vs-architecture-vs-algorithm controls. These gaps are a coordination failure, not a fault of any individual lab; the authors of this paper, like many others in the GFM community, have contributed to them. Rather than just critiquing the community, we aim to provide concrete steps toward a shared understanding of how to innovate GFMs.

地理空间基础模型评估标准可复现性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。