统一检测与描述遥感变化,提升双任务效率
ChangeMinds: Multi-task Framework for Detecting and Describing Changes in Remote Sensing
- 设计联合框架,用时序特征捕捉变化动态
- 在LEVIR-MCI上双任务性能超越现有方法
- 适合需要多任务协同的遥感分析场景
遥感变化检测(CD)与变化描述(CC)近年借助深度学习取得显著进展,但现有方法常独立处理,缺乏协同效应。本文提出ChangeMinds,一种统一的多任务框架,在单个端到端模型中同步优化CD与CC。通过提出变化感知长短期记忆模块(ChangeLSTM),有效捕获双时相深度特征中的复杂时空动态,生成通用的变化感知表示,同时服务于两任务。进一步引入具有交叉注意力机制的多任务预测器,增强图像与文本特征间的交互,促进高效协同学习。在LEVIR-MCI及多个标准基准上的大量实验表明,ChangeMinds在多任务学习设置下优于现有方法,并显著提升单一任务性能。代码与预训练模型将公开。
原文摘要 · Abstract (English)
Recent advancements in Remote Sensing (RS) for Change Detection (CD) and Change Captioning (CC) have seen substantial success by adopting deep learning techniques. Despite these advances, existing methods often handle CD and CC tasks independently, leading to inefficiencies from the absence of synergistic processing. In this paper, we present ChangeMinds, a novel unified multi-task framework that concurrently optimizes CD and CC processes within a single, end-to-end model. We propose the change-aware long short-term memory module (ChangeLSTM) to effectively capture complex spatiotemporal dynamics from extracted bi-temporal deep features, enabling the generation of universal change-aware representations that effectively serve both CC and CD tasks. Furthermore, we introduce a multi-task predictor with a cross-attention mechanism that enhances the interaction between image and text features, promoting efficient simultaneous learning and processing for both tasks. Extensive evaluations on the LEVIR-MCI dataset, alongside other standard benchmarks, show that ChangeMinds surpasses existing methods in multi-task learning settings and markedly improves performance in individual CD and CC tasks. Codes and pre-trained models will be available online.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。