通过优化采样起点,提升无训练风格迁移的布局保持与内容纯净度。
StyleSSP: Sampling StartPoint Enhancement for Training-free Diffusion-based Method for Style Transfer
- 在采样阶段前调整潜在表示的频率成分,强化内容布局保留。
- 利用反演阶段的负向引导,减少风格图像内容泄露。
- 适用于追求高质量无训练风格迁移的视觉生成研究者。
无训练的扩散模型风格迁移方法在避免大量训练的同时取得了显著成果。然而,由于缺乏对风格信息提取的针对性训练以及对内容图像布局的约束,现有方法常出现原始内容布局变化和风格图像内容泄露的问题。通过一系列实验,我们发现采样阶段的有效起始点能显著提升风格迁移效果。基于此,提出StyleSSP,通过两个关键组件解决上述问题:(1) 频率调控:降低DDIM潜在空间中的低频成分,使采样过程更关注内容图像的布局;(2) 反演阶段的负向引导:在反演过程中引入负向指导,确保采样起始点远离风格图像的内容特征。实验表明,StyleSSP在保持原始内容结构和减少风格图像内容泄露方面优于现有无训练基线方法。项目页面:https://github.com/bytedance/StyleSSP。
原文摘要 · Abstract (English)
Training-free diffusion-based methods have achieved remarkable success in style transfer, eliminating the need for extensive training or fine-tuning. However, due to the lack of targeted training for style information extraction and constraints on the content image layout, training-free methods often suffer from layout changes of original content and content leakage from style images. Through a series of experiments, we discovered that an effective startpoint in the sampling stage significantly enhances the style transfer process. Based on this discovery, we propose StyleSSP, which focuses on obtaining a better startpoint to address layout changes of original content and content leakage from style image. StyleSSP comprises two key components: (1) Frequency Manipulation: To improve content preservation, we reduce the low-frequency components of the DDIM latent, allowing the sampling stage to pay more attention to the layout of content images; and (2) Negative Guidance via Inversion: To mitigate the content leakage from style image, we employ negative guidance in the inversion stage to ensure that the startpoint of the sampling stage is distanced from the content of style image. Experiments show that StyleSSP surpasses previous training-free style transfer baselines, particularly in preserving original content and minimizing the content leakage from style image. Project page: https://github.com/bytedance/StyleSSP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。