用视频模型+变化掩码，让遥感图像变化描述更准确
MV-CC: Mask Enhanced Video Model for Remote Sensing Change Caption
- 用现成视频编码器同时提取时空特征，省去复杂融合模块
- 引入变化检测掩码，引导模型聚焦关键变化区域
- 在多个遥感变化描述数据集上表现优于现有方法
遥感图像变化描述（RSICC）旨在为双时相遥感图像生成自然语言描述。由于该任务需同时捕捉空间与时间特征，以往方法普遍采用编码器-融合-解码器架构：先用图像编码器提取空间特征，再通过融合模块整合并提取时间特征，导致融合设计日益复杂。本文提出一种无需人工设计融合模块的新型视频模型范式，构建了面向变化描述的掩码增强视频模型（MV-CC）。具体而言，我们采用现成的视频编码器，直接对双时相图像同步提取时空特征；同时根据任务需求设定变化类型，并利用变化检测（CD）方法获取的掩码，显式引导模型关注重点区域。实验表明，所提方法在多个主流RSICC数据集上均优于当前先进方法。代码已公开于https://github.com/liuruixun/MV-CC。
原文摘要 · Abstract (English)
Remote sensing image change caption (RSICC) aims to provide natural language descriptions for bi-temporal remote sensing images. Since Change Caption (CC) task requires both spatial and temporal features, previous works follow an encoder-fusion-decoder architecture. They use an image encoder to extract spatial features and the fusion module to integrate spatial features and extract temporal features, which leads to increasingly complex manual design of the fusion module. In this paper, we introduce a novel video model-based paradigm without design of the fusion module and propose a Mask-enhanced Video model for Change Caption (MV-CC). Specifically, we use the off-the-shelf video encoder to simultaneously extract the temporal and spatial features of bi-temporal images. Furthermore, the types of changes in the CC are set based on specific task requirements, and to enable the model to better focus on the regions of interest, we employ masks obtained from the Change Detection (CD) method to explicitly guide the CC model. Experimental results demonstrate that our proposed method can obtain better performance compared with other state-of-the-art RSICC methods. The code is available at https://github.com/liuruixun/MV-CC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。