让扩散模型生成图像时精准匹配指定颜色分布
Color Alignment in Diffusion
- 将图像或隐向量投影到条件色彩空间，约束生成过程
- 在保持生成质量与多样性前提下实现像素级颜色控制
- 适合需要精确颜色调控的图像生成场景
扩散模型在生成视觉上吸引人的图像方面展现出巨大潜力。然而，在细粒度层面进行条件控制仍具挑战，例如根据通用颜色模式生成图像像素。现有方法常导致生成内容偏离目标像素条件。为此，我们提出一种新颖的颜色对齐算法，将扩散模型的生成过程限制在给定颜色模式内。具体而言，我们将扩散项（图像样本或隐表示）投影到条件色彩空间，以对齐输入色彩分布。该策略简化了扩散模型在色彩流形内的预测，同时仍保留生成内容的合理结构，从而实现符合目标颜色模式的多样化内容生成。实验表明，我们的方法在颜色条件控制方面达到当前最优性能，且生成质量与多样性与常规扩散模型相当。
原文摘要 · Abstract (English)
Diffusion models have shown great promise in synthesizing visually appealing images. However, it remains challenging to condition the synthesis at a fine-grained level, for instance, synthesizing image pixels following some generic color pattern. Existing image synthesis methods often produce contents that fall outside the desired pixel conditions. To address this, we introduce a novel color alignment algorithm that confines the generative process in diffusion models within a given color pattern. Specifically, we project diffusion terms, either imagery samples or latent representations, into a conditional color space to align with the input color distribution. This strategy simplifies the prediction in diffusion models within a color manifold while still allowing plausible structures in generated contents, thus enabling the generation of diverse contents that comply with the target color pattern. Experimental results demonstrate our state-of-the-art performance in conditioning and controlling of color pixels, while maintaining on-par generation quality and diversity in comparison with regular diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。