构建首个含云干扰的遥感多模态数据集,提升模型在有云场景下的鲁棒性。
CBEN -- A Multimodal Machine Learning Dataset for Cloud Robust Remote Sensing Image Understanding
- 构建光学与雷达图像配对数据集CBEN,包含云遮挡样本用于训练评估
- 主流方法在有云图像上性能下降23-33个百分点,暴露其脆弱性
- 通过在训练中引入云图像,实现17.2-28.7个百分点的相对提升,适合灾情监测等实时应用
云层是影响光学卫星影像的常见现象,给遥感分析带来挑战。现有研究常排除有云图像,导致模型无法适应真实场景,尤其在自然灾害等时间敏感任务中失效。为解决此问题,本文构建了名为CloudyBigEarthNet(CBEN)的多模态数据集,包含光学与雷达图像的配对样本,并包含云遮挡情况,用于训练与评估。以平均精度(AP)为指标,发现当前最优方法在有云图像上的性能下降23-33个百分点。通过在训练中加入有云光学数据,所提方法在有云测试集上获得17.2-28.7个百分点的相对提升。代码与数据集已公开于GitHub。
原文摘要 · Abstract (English)
Clouds are a common phenomenon that distorts optical satellite imagery, which poses a challenge for remote sensing. However, in the literature cloudless analysis is often performed where cloudy images are excluded from machine learning datasets and methods. Such an approach cannot be applied to time sensitive applications, e.g., during natural disasters. A possible solution is to apply cloud removal as a preprocessing step to ensure that cloudfree solutions are not failing under such conditions. But cloud removal methods are still actively researched and suffer from drawbacks, such as generated visual artifacts. Therefore, it is desirable to develop cloud robust methods that are less affected by cloudy weather. Cloud robust methods can be achieved by combining optical data with radar, a modality unaffected by clouds. While many datasets for machine learning combine optical and radar data, most researchers exclude cloudy images. We identify this exclusion from machine learning training and evaluation as a limitation that reduces applicability to cloudy scenarios. To investigate this, we assembled a dataset, named CloudyBigEarthNet (CBEN), of paired optical and radar images with cloud occlusion for training and evaluation. Using average precision (AP) as the evaluation metric, we show that state-of-the-art methods trained on combined clear-sky optical and radar imagery suffer performance drops of 23-33 percentage points when evaluated on cloudy images. We then adapt these methods to cloudy optical data during training, achieving relative improvement of 17.2-28.7 percentage points on cloudy test cases compared with the original approaches. Code and dataset are publicly available at: https://github.com/mstricker13/CBEN
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。