用切片水仙距离引导扩散模型，让生成图像颜色匹配参考图且语义合理。
Color Conditional Generation with Sliced Wasserstein Guidance
- 修改扩散模型采样过程，引入可微分的切片1-Wasserstein距离
- 生成图像颜色与参考图相似度显著提升，同时保持文本提示的语义一致
- 无需训练，适合需要精准配色的图像生成任务
我们提出SW-Guidance，一种无需训练的图像生成方法，可根据参考图像的颜色分布进行条件生成。传统方法先生成图像再应用风格迁移，常导致生成结果颜色语义混乱。本方法通过在扩散模型采样过程中引入生成图像与参考调色板之间可微分的切片1-Wasserstein距离，有效约束颜色分布。实验表明，该方法在颜色相似性上优于现有最优技术，生成图像不仅准确匹配参考颜色，还保持与原始文本提示的语义一致性。代码已开源：https://github.com/alobashev/sw-guidance/
原文摘要 · Abstract (English)
We propose SW-Guidance, a training-free approach for image generation conditioned on the color distribution of a reference image. While it is possible to generate an image with fixed colors by first creating an image from a text prompt and then applying a color style transfer method, this approach often results in semantically meaningless colors in the generated image. Our method solves this problem by modifying the sampling process of a diffusion model to incorporate the differentiable Sliced 1-Wasserstein distance between the color distribution of the generated image and the reference palette. Our method outperforms state-of-the-art techniques for color-conditional generation in terms of color similarity to the reference, producing images that not only match the reference colors but also maintain semantic coherence with the original text prompt. Our source code is available at https://github.com/alobashev/sw-guidance/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。