arXiv:2411.13127cs.CV2024-11被引 14

用轻量模块适配预训练视觉模型,提升遥感云检测精度与鲁棒性

Adapting Vision Foundation Models for Robust Cloud Segmentation in Remote Sensing Images

  • 冻结预训练视觉模型,仅用0.6%参数的轻量模块进行适配
  • 在多源卫星数据上实现领先性能,跨传感器和场景泛化能力强
  • 适合遥感图像处理、云检测研究者快速部署高效模型

云分割是遥感图像解译中的关键挑战,其准确性直接影响后续数据处理与分析效果。近年来,视觉基础模型(VFM)在多种视觉任务中展现出强大的泛化能力。本文提出一种参数高效的自适应方法——Cloud-Adapter,用于提升云分割的准确率与鲁棒性。该方法利用在通用领域预训练的视觉基础模型,保持其冻结状态,无需额外训练。Cloud-Adapter引入一个轻量级空间感知模块,首先通过卷积神经网络(ConvNet)提取密集的空间特征,再将多尺度特征聚合后作为上下文输入至适配模块,调节冻结的Transformer层。实验表明,仅使用冻结主干0.6%的可训练参数,该方法在多个来源卫星数据、不同传感器系列、数据处理层级、地表覆盖场景及标注粒度的云分割数据集上均达到当前最优表现。代码与模型已公开于https://xavierjiezou.github.io/Cloud-Adapter/,以支持后续研究。

原文摘要 · Abstract (English)

Cloud segmentation is a critical challenge in remote sensing image interpretation, as its accuracy directly impacts the effectiveness of subsequent data processing and analysis. Recently, vision foundation models (VFM) have demonstrated powerful generalization capabilities across various visual tasks. In this paper, we present a parameter-efficient adaptive approach, termed Cloud-Adapter, designed to enhance the accuracy and robustness of cloud segmentation. Our method leverages a VFM pretrained on general domain data, which remains frozen, eliminating the need for additional training. Cloud-Adapter incorporates a lightweight spatial perception module that initially utilizes a convolutional neural network (ConvNet) to extract dense spatial representations. These multi-scale features are then aggregated and serve as contextual inputs to an adapting module, which modulates the frozen transformer layers within the VFM. Experimental results demonstrate that the Cloud-Adapter approach, utilizing only 0.6% of the trainable parameters of the frozen backbone, achieves substantial performance gains. Cloud-Adapter consistently achieves state-of-the-art performance across various cloud segmentation datasets from multiple satellite sources, sensor series, data processing levels, land cover scenarios, and annotation granularities. We have released the code and model checkpoints at https://xavierjiezou.github.io/Cloud-Adapter/ to support further research.

遥感图像云分割视觉模型参数高效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。