无需训练即可精准生成指定数量物体的图像。
CountDiffusion: Text-to-Image Synthesis with Training-Free Counting-Guidance Diffusion
- 分两阶段：先生成中间图像并计数，再通过注意力修正数量
- 在多个基准上显著提升图像中物体数量的准确性
- 可无缝接入任意扩散模型，适合需要精确数量控制的场景
Stable Diffusion 推动了文本到图像合成的发展，但训练模型准确生成特定数量物体仍面临计算成本高和抽象数量概念难以传授的挑战。本文提出 CountDiffusion，一种无需训练的框架，旨在根据文本描述生成具有正确物体数量的图像。CountDiffusion 包含两个阶段：第一阶段，通过扩散模型一步去噪生成中间结果，并用计数模型统计其中物体数量；第二阶段，利用修正模块通过调整注意力图来纠正物体数量，实现通用引导。所提方法可直接插入任意基于扩散的文本到图像（T2I）生成模型，无需额外训练。实验表明，CountDiffusion 显著提升了 T2I 模型在物体数量生成上的准确性。
原文摘要 · Abstract (English)
Stable Diffusion has advanced text-to-image synthesis, but training models to generate images with accurate object quantity is still difficult due to the high computational cost and the challenge of teaching models the abstract concept of quantity. In this paper, we propose CountDiffusion, a training-free framework aiming at generating images with correct object quantity from textual descriptions. CountDiffusion consists of two stages. In the first stage, an intermediate denoising result is generated by the diffusion model to predict the final synthesized image with one-step denoising, and a counting model is used to count the number of objects in this image. In the second stage, a correction module is used to correct the object quantity by changing the attention map of the object with universal guidance. The proposed CountDiffusion can be plugged into any diffusion-based text-to-image (T2I) generation models without further training. Experiment results demonstrate the superiority of our proposed CountDiffusion, which improves the accurate object quantity generation ability of T2I models by a large margin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。