融合与分组策略提升多模态遥感城市气候区分类精度
Fusion and Grouping Strategies in Deep Learning for Local Climate Zone Classification of Multimodal Remote Sensing Data
- 设计四种融合方法，结合注意力与多尺度滤波增强特征提取
- 带数据分组的基线融合模型在So2Sat数据集上达76.6%准确率
- 对少数类样本效果显著，适合城市气候研究与遥感分析者
城市气候区（LCZ）为研究城市结构与土地利用、分析城市化对局部气候影响提供分区地图。多模态遥感支持LCZ分类，数据融合对提升精度至关重要，但现有深度学习模型中融合机制缺乏系统分析。本研究针对多分类任务，分析不同融合策略与基于数据特性的分组策略。对比四种卷积神经网络模型：(i) 基线混合融合（FM1），(ii) 自注意力与交叉注意力融合（FM2），(iii) 多尺度高斯滤波图像融合（FM3），(iv) 加权决策层融合（FM4）。通过消融实验评估像素级、特征级与决策级融合的影响。分组策略包括模态内波段分组（BG）与真值标签合并（LM）。所有分析基于So2Sat LCZ42数据集，包含合成孔径雷达（SAR）与多光谱成像（MSI）图像对。结果表明，FM1持续优于简单融合方法；结合BG与LM的FM1表现最优，整体准确率达76.6%。研究强调这些策略对提升少数类预测精度的重要作用。代码与处理数据集见https://github.com/GVCL/LCZC-MultiModalHybridFusion。
原文摘要 · Abstract (English)
Local Climate Zones (LCZs) give a zoning map to study urban structures and land use and analyze the impact of urbanization on local climate. Multimodal remote sensing enables LCZ classification, for which data fusion is significant for improving accuracy owing to the data complexity. However, there is a gap in a comprehensive analysis of the fusion mechanisms used in their deep learning (DL) classifier architectures. This study analyzes different fusion strategies in the multi-class LCZ classification models for multimodal data and grouping strategies based on inherent data characteristics. The different models involving Convolutional Neural Networks (CNNs) include: (i) baseline hybrid fusion (FM1), (ii) with self- and cross-attention mechanisms (FM2), (iii) with the multi-scale Gaussian filtered images (FM3), and (iv) weighted decision-level fusion (FM4). Ablation experiments are conducted to study the pixel-, feature-, and decision-level fusion effects in the model performance. Grouping strategies include band grouping (BG) within the data modalities and label merging (LM) in the ground truth. Our analysis is exclusively done on the So2Sat LCZ42 dataset, which consists of Synthetic Aperture Radar (SAR) and Multispectral Imaging (MSI) image pairs. Our results show that FM1 consistently outperforms simple fusion methods. FM1 with BG and LM is found to be the most effective approach among all fusion strategies, giving an overall accuracy of 76.6\%. Importantly, our study highlights the effect of these strategies in improving prediction accuracy for the underrepresented classes. Our code and processed datasets are available at https://github.com/GVCL/LCZC-MultiModalHybridFusion
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。