arXiv:2412.19179cs.CVcs.AI2024-12被引 36

用扩散模型提升遥感变化描述的准确性和可解释性。

Mask Approximation Net: A Novel Diffusion Model Approach for Remote Sensing Change Captioning

  • 基于扩散模型学习数据分布,替代传统经验式设计。
  • 在多个遥感数据集上优于现有方法,显著提升描述质量。
  • 适合遥感图像分析、多模态生成领域的研究者参考。

遥感图像变化描述是一项创新的多模态任务,不仅能检测地表状态变化,还能提供全面的文字描述,提升人类可读性和交互性。当前深度学习方法通常采用特征提取、特征融合、变化定位和文本生成的三阶段框架,多数工作集中于设计复杂的网络模块,缺乏坚实的理论指导,依赖大量经验实验和迭代调参,易导致过拟合与设计瓶颈,限制模型泛化能力。为此,本文提出一种转向数据分布学习的范式,结合频域噪声过滤的扩散模型,为多模态遥感变化描述提供理论驱动且高效的解决方案。所提方法包含一个简单的多尺度变化检测模块,其输出特征经由精心设计的扩散模型进一步优化,并引入频域引导的复数滤波模块,在扩散过程中有效抑制高频噪声。我们在多个遥感变化检测与描述数据集上验证了该方法的有效性,结果表明其性能优于现有技术。代码将发布于 \\(https://github.com/sundongwei)\\{MaskApproxNet}。

原文摘要 · Abstract (English)

Remote sensing image change description represents an innovative multimodal task within the realm of remote sensing processing.This task not only facilitates the detection of alterations in surface conditions, but also provides comprehensive descriptions of these changes, thereby improving human interpretability and interactivity.Current deep learning methods typically adopt a three stage framework consisting of feature extraction, feature fusion, and change localization, followed by text generation. Most approaches focus heavily on designing complex network modules but lack solid theoretical guidance, relying instead on extensive empirical experimentation and iterative tuning of network components. This experience-driven design paradigm may lead to overfitting and design bottlenecks, thereby limiting the model's generalizability and adaptability.To address these limitations, this paper proposes a paradigm that shift towards data distribution learning using diffusion models, reinforced by frequency-domain noise filtering, to provide a theoretically motivated and practically effective solution to multimodal remote sensing change description.The proposed method primarily includes a simple multi-scale change detection module, whose output features are subsequently refined by a well-designed diffusion model.Furthermore, we introduce a frequency-guided complex filter module to boost the model performance by managing high-frequency noise throughout the diffusion process. We validate the effectiveness of our proposed method across several datasets for remote sensing change detection and description, showcasing its superior performance compared to existing techniques. The code will be available at \href{https://github.com/sundongwei}{MaskApproxNet}.

遥感变化扩散模型多模态生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。