用轻量翻译器让扩散模型高效完成图像转换。
A Diffusion Model Translator for Efficient Image-to-Image Translation
- 在中间步骤直接转移图像分布，跳过全程迭代。
- 自动选择最佳时间步，提升转换质量与速度。
- 适用于风格迁移、上色等任务，效果优于现有方法。
将扩散模型应用于图像到图像翻译（I2I）近期受到广泛关注，因其具有实际应用价值。以往方法在每一步去噪中注入源图像信息以实现逐步优化，导致计算耗时。本文提出一种高效方法，通过引入轻量级翻译器——扩散模型翻译器（DMT），实现 I2I 转换。我们首先从理论上证明，在使用经典 DDPM 框架进行 I2I 任务时，仅需在某个中间步骤完成域间分布转移即可实现有效转换。进一步观察发现，翻译性能高度依赖于所选的时间步，因此提出一种实用策略，可自动为特定任务选择最优时间步。我们在多种 I2I 应用中验证该方法，包括图像风格迁移、颜色化、分割图转图像以及草图转图像，结果表明 DMT 在质量和效率上均优于现有方法。代码将公开。
原文摘要 · Abstract (English)
Applying diffusion models to image-to-image translation (I2I) has recently received increasing attention due to its practical applications. Previous attempts inject information from the source image into each denoising step for an iterative refinement, thus resulting in a time-consuming implementation. We propose an efficient method that equips a diffusion model with a lightweight translator, dubbed a Diffusion Model Translator (DMT), to accomplish I2I. Specifically, we first offer theoretical justification that in employing the pioneering DDPM work for the I2I task, it is both feasible and sufficient to transfer the distribution from one domain to another only at some intermediate step. We further observe that the translation performance highly depends on the chosen timestep for domain transfer, and therefore propose a practical strategy to automatically select an appropriate timestep for a given task. We evaluate our approach on a range of I2I applications, including image stylization, image colorization, segmentation to image, and sketch to image, to validate its efficacy and general utility. The comparisons show that our DMT surpasses existing methods in both quality and efficiency. Code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。