融合视觉与注意力机制，提升洪水淹没区域分割精度
Prithvi-Complimentary Adaptive Fusion Encoder (CAFE): unlocking full-potential for flood inundation mapping
- 用预训练模型+并行卷积分支捕捉局部细节
- 在两个数据集上达到83.41和64.70的IoU新高
- 适合多通道遥感数据的细粒度分割任务
地理基础模型（GFMs）在语义分割、分类和回归等下游任务中表现优异，但在使用Sen1Flood11数据集进行洪水映射时，其性能仍不及基准U-Net，暴露出对关键局部细节捕捉能力不足的问题。为此，我们提出Prithvi-互补自适应融合编码器（CAFE），将预训练的Prithvi GFM编码器与增强卷积注意力模块（CAM）的并行CNN残差分支相结合。该模型通过适配器实现快速高效微调，并完成多尺度、多层次的特征融合，在保留长距离依赖的同时捕捉关键局部信息。在两个综合性洪水映射数据集上均取得领先结果：在Sen1Flood11测试集上，Prithvi-CAFE IoU达83.41，优于原始Prithvi（82.50）、TerraMind（82.90）、DOFA（81.54）和spectralGPT（81.02）；在保留测试站点上，其IoU为81.37，远超基线U-Net（70.57）和原始Prithvi（72.42）。在FloodPlanet数据集上，其IoU达64.70，超过基线U-Net（60.14）、TerraMind（62.33）、DOFA（59.15）及Prithvi 2.0（61.91）。所提方法简单有效，适用于多通道、多模态数据中需精细分割的任务。代码已开源。
原文摘要 · Abstract (English)
Geo-Foundation Models (GFMs), have proven effective in diverse downstream applications, including semantic segmentation, classification, and regression tasks. However, in case of flood mapping using Sen1Flood11 dataset as a downstream task, GFMs struggles to outperform the baseline U-Net, highlighting model's limitation in capturing critical local nuances. To address this, we present the Prithvi-Complementary Adaptive Fusion Encoder (CAFE), which integrate Prithvi GFM pretrained encoder with a parallel CNN residual branch enhanced by Convolutional Attention Modules (CAM). Prithvi-CAFE enables fast and efficient fine-tuning through adapters in Prithvi and performs multi-scale, multi-level fusion with CNN features, capturing critical local details while preserving long-range dependencies. We achieve state-of-the-art results on two comprehensive flood mapping datasets: Sen1Flood11 and FloodPlanet. On Sen1Flood11 test data, Prithvi-CAFE (IoU 83.41) outperforms the original Prithvi (IoU 82.50) and other major GFMs (TerraMind 82.90, DOFA 81.54, spectralGPT: 81.02). The improvement is even more pronounced on the hold-out test site, where Prithvi-CAFE achieves an IoU of 81.37 compared to the baseline U-Net (70.57) and original Prithvi (72.42). On FloodPlanet, Prithvi-CAFE also surpasses the baseline U-Net and other GFMs, achieving an IoU of 64.70 compared to U-Net (60.14), Terramind (62.33), DOFA (59.15) and Prithvi 2.0 (61.91). Our proposed simple yet effective Prithvi-CAFE demonstrates strong potential for improving segmentation tasks where multi-channel and multi-modal data provide complementary information and local details are critical. The code is released on \href{https://github.com/Sk-2103/Prithvi-CAFE}{Prithvi-CAFE Github}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。