arXiv:2603.14132cs.CVcs.LG2026-03

用双路Transformer融合多模态数据,提升火星滑坡自动分割精度。

DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++

  • 双路Swin Transformer分别处理可见光与地质数据,分层提取特征。
  • 多尺度融合+UNet++解码器,使边界更清晰,测试集mIoU达0.783。
  • 适合行星科学、遥感分析与小样本多模态学习研究者。

自动化分割火星滑坡,尤其在瓦勒斯马里纳里斯等构造活跃区,对行星地质学、灾害评估和未来机器人探测至关重要。然而,由于可用传感模态异质性强且标注样本有限,从行星图像中检测滑坡极具挑战。每组观测结合了RGB影像与数字高程模型、坡度图、热惯性及上下文灰度影像,其分辨率与统计特性差异显著。为此,我们提出DualSwinFusionSeg,一种多模态分割架构,通过分离模态特异性特征提取并实现多尺度跨模态融合。模型采用两个并行的Swin Transformer V2编码器,分别独立处理RGB与辅助地质输入,生成层次化特征表示。两路对应特征在多尺度进行融合,并由带密集嵌套跳跃连接的UNet++解码器解码,以保留精细边界细节。大量消融实验评估了模态贡献、损失函数、解码器结构与融合策略。在PBVS 2026火星滑坡挑战赛的MMLSv2数据集上的实验表明,模态特异性编码器与基于简单拼接的融合策略在少量训练数据下提升分割精度。最终模型在开发基准上达到0.867 mIoU与0.905 F1,持留测试集上为0.783 mIoU,展现了强多模态行星表面分割性能。

原文摘要 · Abstract (English)

Automated segmentation of Martian landslides, particularly in tectonically active regions such as Valles Marineris,is important for planetary geology, hazard assessment, and future robotic exploration. However, detecting landslides from planetary imagery is challenging due to the heterogeneous nature of available sensing modalities and the limited number of labeled samples. Each observation combines RGB imagery with geophysical measurements such as digital elevation models, slope maps, thermal inertia, and contextual grayscale imagery, which differ significantly in resolution and statistical properties. To address these challenges, we propose DualSwinFusionSeg, a multimodal segmentation architecture that separates modality-specific feature extraction and performs multi-scale cross-modal fusion. The model employs two parallel Swin Transformer V2 encoders to independently process RGB and auxiliary geophysical inputs, producing hierarchical feature representations. Corresponding features from the two streams are fused at multiple scales and decoded using a UNet++ decoder with dense nested skip connections to preserve fine boundary details. Extensive ablation studies evaluate modality contributions, loss functions, decoder architectures, and fusion strategies. Experiments on the MMLSv2 dataset from the PBVS 2026 Mars-LS Challenge show that modality-specific encoders and simple concatenation-based fusion improve segmentation accuracy under limited training data. The final model achieves 0.867 mIoU and 0.905 F1 on the development benchmark and 0.783 mIoU on the held-out test set, demonstrating strong performance for multimodal planetary surface segmentation.

火星滑坡多模态分割Swin TransformerUNet++

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。