OlmoEarth用新方法建模地球观测数据，提升多模态分析性能。
OlmoEarth: Stable Latent Image Modeling for Multimodal Earth Observation
- 设计专用于地球观测的自监督学习框架与掩码策略。
- 在24项任务中15项嵌入效果最优，29项微调任务中19项领先。
- 开源模型和平台，助力非营利组织解决全球问题。
地球观测数据兼具图像的空间性、视频或文本的时序性以及高度多模态特性。我们提出OlmoEarth：一个面向地球观测领域的多模态时空基础模型，采用专为该领域设计的新颖自监督学习范式、掩码策略和损失函数。相比12个其他基础模型，OlmoEarth在多个研究基准和外部合作方的实际任务中均达到最先进水平。评估嵌入表现时，在24项任务中有15项最优；全量微调后，在29项任务中有19项领先。我们将OlmoEarth部署为端到端平台的核心，支持地球观测模型的数据采集、标注、训练与推理。该平台将前沿基础模型与强大数据管理工具提供给致力于解决全球重大问题的非营利组织。OlmoEarth的源代码、训练数据及预训练权重已公开于https://github.com/allenai/olmoearth_pretrain。
原文摘要 · Abstract (English)
Earth observation data presents a unique challenge: it is spatial like images, sequential like video or text, and highly multimodal. We present OlmoEarth: a multimodal, spatio-temporal foundation model that employs a novel self-supervised learning formulation, masking strategy, and loss all designed for the Earth observation domain. OlmoEarth achieves state-of-the-art performance compared to 12 other foundation models across a variety of research benchmarks and real-world tasks from external partners. When evaluating embeddings OlmoEarth achieves the best performance on 15 out of 24 tasks, and with full fine-tuning it is the best on 19 of 29 tasks. We deploy OlmoEarth as the backbone of an end-to-end platform for data collection, labeling, training, and inference of Earth observation models. The OlmoEarth Platform puts frontier foundation models and powerful data management tools into the hands of non-profits and NGOs working to solve the world's biggest problems. OlmoEarth source code, training data, and pre-trained weights are available at $\href{https://github.com/allenai/olmoearth_pretrain}{\text{https://github.com/allenai/olmoearth_pretrain}}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。