首个评估遥感大模型鲁棒性的基准,揭示现有模型在真实干扰下表现脆弱。
REOBench: Benchmarking Robustness of Earth Observation Foundation Models
- 构建涵盖12类图像畸变的遥感图像基准,覆盖6个任务
- 模型性能下降最高超20%,不同任务与模型差异显著
- 多模态模型鲁棒性更强,适合灾害监测等高可靠性场景
地球观测基础模型在多项任务中展现出强大泛化能力,但其在真实世界扰动下的鲁棒性尚未充分探索。为此,我们提出REOBench,首个全面评估地球观测基础模型鲁棒性的基准,覆盖六项任务和十二种图像畸变,包括外观与几何扰动。为确保评估真实且精细,基准聚焦于高分辨率光学遥感图像,广泛应用于城市规划与灾害响应等关键场景。我们系统评估了采用掩码图像建模、对比学习和视觉-语言预训练范式的多种模型。结果表明:(1)现有地球观测基础模型在输入畸变下出现显著性能下降;(2)退化程度因任务、模型架构、主干规模和畸变类型而异,性能下降幅度从不足1%到超过20%;(3)视觉-语言模型在多模态任务中表现出更强鲁棒性。REOBench揭示了当前地球观测基础模型对真实世界畸变的脆弱性,并为构建更鲁棒可靠的模型提供可行动见解。代码与数据公开于https://github.com/lx709/REOBench。
原文摘要 · Abstract (English)
Earth observation foundation models have shown strong generalization across multiple Earth observation tasks, but their robustness under real-world perturbations remains underexplored. To bridge this gap, we introduce REOBench, the first comprehensive benchmark for evaluating the robustness of Earth observation foundation models across six tasks and twelve types of image corruptions, including both appearance-based and geometric perturbations. To ensure realistic and fine-grained evaluation, our benchmark focuses on high-resolution optical remote sensing images, which are widely used in critical applications such as urban planning and disaster response. We conduct a systematic evaluation of a broad range of models trained using masked image modeling, contrastive learning, and vision-language pre-training paradigms. Our results reveal that (1) existing Earth observation foundation models experience significant performance degradation when exposed to input corruptions. (2) The severity of degradation varies across tasks, model architectures, backbone sizes, and types of corruption, with performance drop varying from less than 1% to over 20%. (3) Vision-language models show enhanced robustness, particularly in multimodal tasks. REOBench underscores the vulnerability of current Earth observation foundation models to real-world corruptions and provides actionable insights for developing more robust and reliable models. Code and data are publicly available at https://github.com/lx709/REOBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。