arXiv:2504.17822cs.CVcs.AI2025-04被引 7

用多尺度视觉Transformer融合多模态数据,精准识别北极冻土退化的滑塌地貌。

A multi-scale vision transformer-based multimodal GeoAI model for mapping Arctic permafrost thaw

  • 采用多尺度视觉Transformer与残差跨模态注意力融合特征
  • 在多个数据集上准确率超现有方法,尤其在边界模糊区域表现优异
  • 适合遥感、环境监测领域研究者参考,降低算力成本

北极地区退化型冻土滑塌(Retrogressive Thaw Slumps, RTS)是具有显著环境影响的冻土地貌,其出现是冻土退化的明确信号。然而,由于尺度小、边界模糊及时空变化性强,精确检测面临挑战。本文采用基于多尺度视觉Transformer的级联掩码R-CNN模型,提出两种新策略优化多模态学习:(1) 特征级残差跨模态注意力融合,有效整合多源特征图以捕捉互补信息,增强对复杂模式的理解;(2) 先单模态预训练再多模态微调,缓解高计算需求同时保持高性能。实验表明,该方法优于采用数据级融合、卷积特征融合及多种注意力融合策略的现有模型,在多个测试集上实现更优精度,为高效利用多模态数据进行RTS制图提供了新思路,有助于深化对冻土地貌及其环境影响的认知。

原文摘要 · Abstract (English)

Retrogressive Thaw Slumps (RTS) in Arctic regions are distinct permafrost landforms with significant environmental impacts. Mapping these RTS is crucial because their appearance serves as a clear indication of permafrost thaw. However, their small scale compared to other landform features, vague boundaries, and spatiotemporal variation pose significant challenges for accurate detection. In this paper, we employed a state-of-the-art deep learning model, the Cascade Mask R-CNN with a multi-scale vision transformer-based backbone, to delineate RTS features across the Arctic. Two new strategies were introduced to optimize multimodal learning and enhance the model's predictive performance: (1) a feature-level, residual cross-modality attention fusion strategy, which effectively integrates feature maps from multiple modalities to capture complementary information and improve the model's ability to understand complex patterns and relationships within the data; (2) pre-trained unimodal learning followed by multimodal fine-tuning to alleviate high computing demand while achieving strong model performance. Experimental results demonstrated that our approach outperformed existing models adopting data-level fusion, feature-level convolutional fusion, and various attention fusion strategies, providing valuable insights into the efficient utilization of multimodal data for RTS mapping. This research contributes to our understanding of permafrost landforms and their environmental implications.

冻土监测多模态融合视觉TransformerGeoAI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。