arXiv:2501.10197cs.CV2025-01被引 6

提出CSHNet解决雷达图转光学图等信息不对称图像转换难题

CSHNet: A Novel Information Asymmetric Image Translation Method

  • 设计CNN与Swin Transformer混合结构,兼顾细节与全局结构
  • 在场景级和实例级数据集上均超越现有方法,提升视觉质量与指标
  • 适合关注跨域图像生成、结构保持任务的研究者

尽管跨域图像转换取得进展,但在如合成孔径雷达到光学图像、草图到实例等信息不对称任务中仍面临挑战,这类任务需将低细节域数据转化为高内容丰富度域。传统基于CNN的方法虽能捕捉细粒度特征,但难以保持全局结构,导致图像区域错误合并。为此,我们提出CNN-Swin混合网络(CSHNet),包含两个核心模块:嵌入Swin的CNN(SEC)和嵌入CNN的Swin(CES),构成SEC-CES-瓶颈(SCB)。SEC利用CNN提取细节并融合Swin的结构先验;CES则保留Swin的全局完整性,弥补CNN对结构关注不足。此外,引入交互引导连接(IGC)实现SEC与CES间动态信息交换,以及自适应边缘感知损失(AEPL)以维持转换过程中的结构边界。实验表明,CSHNet在场景级和实例级数据集上均优于现有方法,在视觉质量和性能指标上均有提升。代码已开源:https://github.com/XduShi/CSHNet。

原文摘要 · Abstract (English)

Despite advancements in cross-domain image translation, challenges persist in asymmetric tasks such as SAR-to-Optical and Sketch-to-Instance conversions, which involve transforming data from a less detailed domain into one with richer content. Traditional CNN-based methods are effective at capturing fine details but struggle with global structure, leading to unwanted merging of image regions. To address this, we propose the CNN-Swin Hybrid Network (CSHNet), which combines two key modules: Swin Embedded CNN (SEC) and CNN Embedded Swin (CES), forming the SEC-CES-Bottleneck (SCB). SEC leverages CNN's detailed feature extraction while integrating the Swin Transformer's structural bias. CES, in turn, preserves the Swin Transformer's global integrity, compensating for CNN's lack of focus on structure. Additionally, CSHNet includes two components designed to enhance cross-domain information retention: the Interactive Guided Connection (IGC), which enables dynamic information exchange between SEC and CES, and Adaptive Edge Perception Loss (AEPL), which maintains structural boundaries during translation. Experimental results show that CSHNet outperforms existing methods in both visual quality and performance metrics across scene-level and instance-level datasets. Our code is available at: https://github.com/XduShi/CSHNet.

图像转换混合网络结构保持SwinTransformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。