arXiv:2503.14275cs.CV2025-03NeurIPS被引 8

无需调参即可分离图像颜色与纹理,实现精准风格化生成。

Free-Lunch Color-Texture Disentanglement for Stylized Image Generation

  • 利用CLIP空间的图像提示可加性,从参考图中解耦颜色和纹理嵌入。
  • 通过正则化白化着色变换,颜色对齐准确率提升,纹理损失减少。
  • 适合需要精细控制风格元素的研究者或创意设计应用。

基于扩散模型的文生图技术在仅用少量风格参考图的情况下实现了显著的风格化生成进展。然而,现有方法在多风格属性(如颜色、纹理)的细粒度控制上仍面临挑战。本文提出首个无需调参的颜色-纹理解耦方法,解决解耦风格图像生成(DisIG)问题。通过利用CLIP图像嵌入空间中的图像提示可加性,从单独的颜色和纹理参考图中分离并提取颜色-纹理嵌入(CTE)。为确保生成图像颜色与参考图一致,采用白化与着色变换增强颜色一致性;同时引入噪声项,缓解扩散训练固有的信号泄漏偏倚,保护纹理保真度。所提出的风格属性解耦方法(SADis)在WikiArt与StyleDrop数据集上的实验表明,其在定性和定量评估上均优于当前最优风格化方法。代码已开源:https://deepffff.github.io/sadis.github.io/。

原文摘要 · Abstract (English)

Recent advances in Text-to-Image (T2I) diffusion models have transformed image generation, enabling significant progress in stylized generation using only a few style reference images. However, current diffusion-based methods struggle with fine-grained style customization due to challenges in controlling multiple style attributes, such as color and texture. This paper introduces the first tuning-free approach to achieve free-lunch color-texture disentanglement in stylized T2I generation, addressing the need for independently controlled style elements for the Disentangled Stylized Image Generation (DisIG) problem. Our approach leverages the Image-Prompt Additivity property in the CLIP image embedding space to develop techniques for separating and extracting Color-Texture Embeddings (CTE) from individual color and texture reference images. To ensure that the color palette of the generated image aligns closely with the color reference, we apply a whitening and coloring transformation to enhance color consistency. Additionally, to prevent texture loss due to the signal-leak bias inherent in diffusion training, we introduce a noise term that preserves textural fidelity during the Regularized Whitening and Coloring Transformation (RegWCT). Through these methods, our Style Attributes Disentanglement approach (SADis) delivers a more precise and customizable solution for stylized image generation. Experiments on images from the WikiArt and StyleDrop datasets demonstrate that, both qualitatively and quantitatively, SADis surpasses state-of-the-art stylization methods in the DisIG task.Code is released at https://deepffff.github.io/sadis.github.io/.

风格化生成颜色纹理解耦扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。