梳理图像翻译中内容保留程度的三类任务,提供系统性对比与评测基准。
Unpaired Image-to-Image Translation with Content Preserving Perspective: A Review
- 按内容保留程度将图像翻译分为全保留、部分保留、非保留三类
- 分析近70种模型,涵盖30个任务和10余项评估指标
- 为仿真到真实图像转换提供可复用的评测基准
图像到图像翻译(I2I)旨在将源域图像转换为目标域图像的同时保持源图内容。该技术广泛应用于风格迁移、图像分割和照片增强等任务。根据应用场景需求,内容保留程度可分为完全保留、部分保留与不保留三类。本文对三类任务分别梳理了代表性方法、数据集及结果,并基于模型架构对I2I方法进行分类研究。同时介绍了领域内十余项主流评估指标。文中分析了近70种I2I模型,覆盖超过30个具体任务与数据集。特别地,针对仿真到真实图像的转换问题,提出一个可复用的基准测试平台。结论表明,不同应用对内容保留的要求各异,应据此选择合适模型。
原文摘要 · Abstract (English)
Image-to-image translation (I2I) transforms an image from a source domain to a target domain while preserving source content. Most computer vision applications are in the field of image-to-image translation, such as style transfer, image segmentation, and photo enhancement. The degree of preservation of the content of the source images in the translation process can be different according to the problem and the intended application. From this point of view, in this paper, we divide the different tasks in the field of image-to-image translation into three categories: Fully Content preserving, Partially Content preserving, and Non-Content preserving. We present different tasks, datasets, methods, results of methods for these three categories in this paper. We make a categorization for I2I methods based on the architecture of different models and study each category separately. In addition, we introduce well-known evaluation criteria in the I2I translation field. Specifically, nearly 70 different I2I models were analyzed, and more than 10 quantitative evaluation metrics and 30 distinct tasks and datasets relevant to the I2I translation problem were both introduced and assessed. Translating from simulation to real images could be well viewed as an application of fully content preserving or partially content preserving unsupervised image-to-image translation methods. So, we provide a benchmark for Sim-to-Real translation, which can be used to evaluate different methods. In general, we conclude that because of the different extent of the obligation to preserving content in various applications, it is better to consider this issue in choosing a suitable I2I model for a specific application.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。