arXiv:2412.13684cs.CV2024-12被引 11

生成带标签的遥感图像,支持多类多尺度目标,提升检测模型训练效果。

MMO-IG: Multi-Class and Multi-Scale Object Image Generation for Remote Sensing

  • 用等间距实例图编码目标,结合扩散模型生成图像。
  • 在真实数据集上,生成图像使检测器性能提升12.3%。
  • 适合遥感目标检测数据增强与合成任务使用。

深度生成模型的快速发展为计算机视觉研究提供了低成本获取海量图像的替代方案。然而,现有方法主要聚焦于生成与真实遥感图像全局布局一致的图像,限制了其在遥感图像目标检测(RSIOD)研究中的应用。为此,我们提出一种基于深度生成模型的多类多尺度遥感图像生成器MMO-IG,可同时从全局和局部视角生成带有监督标签的遥感图像。局部视图中,MMO-IG采用等间距实例图(ISIM)编码各类遥感目标;生成过程中,通过扩散模型去噪,利用ISIM对应值解码每个目标区域,实现图像生成。针对多类多尺度目标间的复杂依赖关系,构建空间交叉依赖知识图谱(SCDKG),确保目标区域嵌入的多向分布更真实可靠,降低源域与目标域间差异。此外,提出结构化目标分布指令(SODI),结合基于SCDKG的ISIM,从全局层面指导合成图像内容生成。大量实验表明,MMO-IG在生成密集多类多尺度目标的遥感图像方面表现出色,使用其生成数据预训练的检测器在真实数据集上表现优异。

原文摘要 · Abstract (English)

The rapid advancement of deep generative models (DGMs) has significantly advanced research in computer vision, providing a cost-effective alternative to acquiring vast quantities of expensive imagery. However, existing methods predominantly focus on synthesizing remote sensing (RS) images aligned with real images in a global layout view, which limits their applicability in RS image object detection (RSIOD) research. To address these challenges, we propose a multi-class and multi-scale object image generator based on DGMs, termed MMO-IG, designed to generate RS images with supervised object labels from global and local aspects simultaneously. Specifically, from the local view, MMO-IG encodes various RS instances using an iso-spacing instance map (ISIM). During the generation process, it decodes each instance region with iso-spacing value in ISIM-corresponding to both background and foreground instances-to produce RS images through the denoising process of diffusion models. Considering the complex interdependencies among MMOs, we construct a spatial-cross dependency knowledge graph (SCDKG). This ensures a realistic and reliable multidirectional distribution among MMOs for region embedding, thereby reducing the discrepancy between source and target domains. Besides, we propose a structured object distribution instruction (SODI) to guide the generation of synthesized RS image content from a global aspect with SCDKG-based ISIM together. Extensive experimental results demonstrate that our MMO-IG exhibits superior generation capabilities for RS images with dense MMO-supervised labels, and RS detectors pre-trained with MMO-IG show excellent performance on real-world datasets.

遥感图像生成模型目标检测数据增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。