arXiv:2602.20721cs.CV2026-02被引 2

解决文生图风格迁移中的内容泄露问题,让风格更纯净。

CleanStyle: Plug-and-Play Style Conditioning Purification for Text-to-Image Stylization

  • 用SVD分解提取风格嵌入中的冗余成分并动态抑制
  • 显著降低内容泄露,提升提示词对齐度与图像质量
  • 无需重训练,可直接接入现有文生图模型

扩散模型中的风格迁移可通过注入参考图像的风格实现可控生成。然而,现有基于编码器的方法虽高效且免调参,常出现内容泄露问题,即风格图像中的语义元素意外出现在输出中,损害提示词忠实度和风格一致性。本文提出CleanStyle,一种无需重训练的即插即用框架,通过奇异值分解(SVD)识别并过滤风格嵌入中的尾部成分(主要携带内容信息)。我们设计了时间感知的指数衰减策略(CS-SVD),在去噪过程中动态抑制这些成分,获得纯净的风格条件嵌入。此外,提出风格特定无分类器指导(SS-CFG),将被抑制的尾部成分重组为风格相关但与提示无关的负向信号,替代传统零向量,有效压制干扰图案。实验表明,CleanStyle在多种风格参考和提示下显著减少内容泄露,提升风格化质量与提示对齐度。

原文摘要 · Abstract (English)

Style transfer in diffusion models enables controllable visual generation by injecting the style of a reference image. However, recent encoder-based methods, while efficient and tuning-free, often suffer from content leakage, where semantic elements from the style image undesirably appear in the output, impairing prompt fidelity and stylistic consistency. In this work, we introduce CleanStyle, a plug-and-play framework that filters out content-related noise from the style embedding without retraining. Motivated by empirical analysis, we observe that such leakage predominantly stems from the tail components of the style embedding, which are isolated via Singular Value Decomposition (SVD). To address this, we propose CleanStyleSVD (CS-SVD), which dynamically suppresses tail components using a time-aware exponential schedule, providing clean, style-preserving conditional embeddings throughout the denoising process. Furthermore, we present Style-Specific Classifier-Free Guidance (SS-CFG), which reuses the suppressed tail components to construct style-aware unconditional inputs. Unlike conventional methods that use generic negative embeddings (e.g., zero vectors), SS-CFG introduces targeted negative signals that reflect style-specific but prompt-irrelevant visual elements. This enables the model to effectively suppress these distracting patterns during generation, thereby improving prompt fidelity and enhancing the overall visual quality of stylized outputs. Our approach is lightweight, interpretable, and can be seamlessly integrated into existing encoder-based diffusion models without retraining. Extensive experiments demonstrate that CleanStyle substantially reduces content leakage, improves stylization quality and improves prompt alignment across a wide range of style references and prompts.

文生图风格迁移扩散模型去噪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。