无需训练即可快速实现图像风格迁移，兼容多种扩散模型。
Harnessing the Latent Diffusion Model for Training-Free Image Style Transfer
- 在反向扩散过程中利用自适应归一化追踪风格编码，实现无训练风格迁移。
- 在潜在空间操作，计算成本低，支持多种预训练扩散模型。
- 速度快、兼容性好，适合快速尝试多种风格与模型组合。
扩散模型近期展现出生成高质量图像的能力，但控制生成过程仍具挑战。图像风格迁移任务旨在将一张风格图的视觉特征迁移到内容图上。传统方法需对预训练模型进行额外训练，带来成本。本文提出一种无需训练的风格迁移算法——风格追踪反向扩散过程（STRDP），适用于预训练的潜在扩散模型（LDM）。该算法在LDM的反向扩散过程中，以独特方式引入自适应实例归一化（AdaIN），同时追踪风格图像的编码历史。该方法在LDM的潜在空间中实现风格迁移，显著降低计算开销，并具备对多种LDM模型的兼容性。通过一系列实验和用户研究，验证了本方法可在不进行额外训练的前提下快速完成图像风格迁移。其高速度、高兼容性及无训练特性，为风格与模型组合的敏捷实验提供了可能，适用于广泛应用场景。
原文摘要 · Abstract (English)
Diffusion models have recently shown the ability to generate high-quality images. However, controlling its generation process still poses challenges. The image style transfer task is one of those challenges that transfers the visual attributes of a style image to another content image. Typical obstacle of this task is the requirement of additional training of a pre-trained model. We propose a training-free style transfer algorithm, Style Tracking Reverse Diffusion Process (STRDP) for a pretrained Latent Diffusion Model (LDM). Our algorithm employs Adaptive Instance Normalization (AdaIN) function in a distinct manner during the reverse diffusion process of an LDM while tracking the encoding history of the style image. This algorithm enables style transfer in the latent space of LDM for reduced computational cost, and provides compatibility for various LDM models. Through a series of experiments and a user study, we show that our method can quickly transfer the style of an image without additional training. The speed, compatibility, and training-free aspect of our algorithm facilitates agile experiments with combinations of styles and LDMs for extensive application.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。