用梯度方向直方图增强无监督图像翻译，减少幻觉、提升质量。
Enhanced Unsupervised Image-to-Image Translation Using Contrastive Learning and Histogram of Oriented Gradients
- 引入HOG特征约束生成图像的结构一致性
- 在无配对数据下显著降低生成幻觉
- 适合需要高质量图像转换的视觉任务
图像到图像翻译是计算机视觉中的关键领域，旨在将图像从一个视觉域转换到另一个域的同时保持其核心内容与结构。然而，该领域面临两大挑战：一是两域数据通常无配对，难以有效训练生成对抗网络；二是现有方法在生成过程中常产生伪影或幻觉，导致图像质量下降。为此，本文提出一种基于对比无配对翻译（CUT）模型的增强型无监督图像翻译方法，引入梯度方向直方图（HOG）特征。该方法通过最小化输入图像与生成图像间HOG特征的损失，即使在无语义标签的情况下也能有效保留图像的语义结构。实验在将GTA5数据集的合成游戏环境转换为Cityscapes数据集的真实城市场景时验证了该方法的有效性，显著减少了生成幻觉并提升了图像质量。
原文摘要 · Abstract (English)
Image-to-Image Translation is a vital area of computer vision that focuses on transforming images from one visual domain to another while preserving their core content and structure. However, this field faces two major challenges: first, the data from the two domains are often unpaired, making it difficult to train generative adversarial networks effectively; second, existing methods tend to produce artifacts or hallucinations during image generation, leading to a decline in image quality. To address these issues, this paper proposes an enhanced unsupervised image-to-image translation method based on the Contrastive Unpaired Translation (CUT) model, incorporating Histogram of Oriented Gradients (HOG) features. This novel approach ensures the preservation of the semantic structure of images, even without semantic labels, by minimizing the loss between the HOG features of input and generated images. The method was tested on translating synthetic game environments from GTA5 dataset to realistic urban scenes in cityscapes dataset, demonstrating significant improvements in reducing hallucinations and enhancing image quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。