arXiv:2503.19798cs.CVeess.IV2025-03被引 2

用关键点引导扩散模型,实现无配对飞机SAR转光学图像。

Unpaired Object-Level SAR-to-Optical Image Translation for Aircraft with Keypoints-Guided Diffusion Models

  • 通过关键点监督类别和航向角,指导扩散模型生成。
  • 在未配对数据下仍保持高保真度,零样本泛化能力出色。
  • 无需人工标注,适合复杂场景下的飞机识别任务。

合成孔径雷达(SAR)图像具备全天候、全时段、高分辨率成像能力,但其独特成像机制导致解读高度依赖专家知识,限制了可解释性,尤其在复杂目标任务中。将SAR图像翻译为光学图像是提升解读性并支持下游任务的可行方案。现有研究多集中于场景级翻译,对象级翻译因缺乏配对数据且难以准确保留轮廓与纹理细节而进展有限。本文提出一种关键点引导的扩散模型(KeypointDiff),用于无配对飞机目标的SAR-to-光学图像翻译。该框架通过关键点引入目标类别与方位角的监督,并设计针对无配对数据的训练策略。基于无分类器引导扩散架构,构建类别-角度引导模块(CAGM),将类别与角度信息融入生成过程。同时采用对抗损失与一致性损失,提升图像保真度与细节质量,专为飞机目标优化。采样阶段借助预训练的关键点检测器,无需人工标注类别与方位角,实现自动化翻译。实验表明,所提方法在多个指标上优于现有方法,为对象级SAR-to-光学翻译及下游任务提供高效有效解决方案。此外,在预训练关键点检测器辅助下,该方法对未训练机型展现出强零样本泛化能力。

原文摘要 · Abstract (English)

Synthetic Aperture Radar (SAR) imagery provides all-weather, all-day, and high-resolution imaging capabilities but its unique imaging mechanism makes interpretation heavily reliant on expert knowledge, limiting interpretability, especially in complex target tasks. Translating SAR images into optical images is a promising solution to enhance interpretation and support downstream tasks. Most existing research focuses on scene-level translation, with limited work on object-level translation due to the scarcity of paired data and the challenge of accurately preserving contour and texture details. To address these issues, this study proposes a keypoint-guided diffusion model (KeypointDiff) for SAR-to-optical image translation of unpaired aircraft targets. This framework introduces supervision on target class and azimuth angle via keypoints, along with a training strategy for unpaired data. Based on the classifier-free guidance diffusion architecture, a class-angle guidance module (CAGM) is designed to integrate class and angle information into the diffusion generation process. Furthermore, adversarial loss and consistency loss are employed to improve image fidelity and detail quality, tailored for aircraft targets. During sampling, aided by a pre-trained keypoint detector, the model eliminates the requirement for manually labeled class and azimuth information, enabling automated SAR-to-optical translation. Experimental results demonstrate that the proposed method outperforms existing approaches across multiple metrics, providing an efficient and effective solution for object-level SAR-to-optical translation and downstream tasks. Moreover, the method exhibits strong zero-shot generalization to untrained aircraft types with the assistance of the keypoint detector.

SAR转光学扩散模型关键点引导无配对翻译

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。