让图像翻译模型自动保持旋转对称性,提升生成质量。
Image-to-Image Translation Framework Embedded with Rotation Symmetry Priors

- 引入旋转等变卷积,让网络天然保持图像旋转对称性
- 在多个任务上优于现有方法,尤其在低配对数据下表现更稳
- 适合对对称性敏感的科学图像生成与医学影像翻译
图像到图像翻译(I2I)是计算机视觉中的基础任务,旨在将源域图像映射到目标域对应图像,同时保留域不变特征并适应域特定属性。尽管基于深度学习的I2I方法取得显著进展,但缺乏成对数据和无监督学习框架仍限制其性能。本文通过将变换对称性先验嵌入I2I网络来解决该问题。具体而言,我们引入旋转群等变卷积,构建首个已知的旋转等变I2I框架,确保自然与科学图像中最本质的旋转对称性在整个网络中得以保持。此外,我们在真实数据集上系统研究了图像对称性先验,并提出一种可学习变换的等变卷积(TL-Conv),可自适应学习变换群,增强跨多样化数据集的对称性保持能力。我们还提供了对TL-Conv等变误差的理论分析,证明其在连续域中保持精确等变性,在离散情况下给出误差上界。通过在多种I2I任务上的广泛实验,验证了所提方法的有效性与优越性能,凸显了等变网络在提升生成质量方面的潜力及其广泛应用前景。代码已开源。
原文摘要 · Abstract (English)
Image-to-image translation (I2I) is a fundamental task in computer vision, focused on mapping an input image from a source domain to a corresponding image in a target domain while preserving domain-invariant features and adapting domain-specific attributes. Despite the remarkable success of deep learning-based I2I approaches, the lack of paired data and unsupervised learning framework still hinder their effectiveness. In this work, we address the challenge by incorporating transformation symmetry priors into image-to-image translation networks. Specifically, we introduce rotation group equivariant convolutions to achieve rotation equivariant I2I framework, a novel contribution, to the best of our knowledge, along this research direction. This design ensures the preservation of rotation symmetry, one of the most intrinsic and domain-invariant properties of natural and scientific images, throughout the network. Furthermore, we conduct a systematic study on image symmetry priors on real dataset and propose a novel transformation learnable equivariant convolutions (TL-Conv) that adaptively learns transformation groups, enhancing symmetry preservation across diverse datasets. We also provide a theoretical analysis of the equivariance error of TL-Conv, proving that it maintains exact equivariance in continuous domains and provide a bound for the error in discrete cases. Through extensive experiments across a range of I2I tasks, we validate the effectiveness and superior performance of our approach, highlighting the potential of equivariant networks in enhancing generation quality and its broad applicability. Our code is available at https://github.com/tanfy929/Equivariant-I2I
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。