让图像不同区域按需去噪,提升文本生成图像的对齐效果。
Asynchronous Denoising Diffusion Models for Aligning Text-to-Image Generation
- 为每个像素分配独立去噪时间步,动态调节去噪节奏。
- 在多种提示下显著提升图像与文本的对齐程度。
- 适合需要高精度图文一致性的生成任务使用。
扩散模型在生成高质量图像方面取得了显著成果,但常难以忠实对齐输入提示。这源于同步去噪机制:所有像素同时从噪声演化为清晰图像,导致提示相关区域在相同噪声水平下仅能参考无关区域,缺乏清晰上下文,影响图文对齐。为此,我们提出异步扩散模型——一种新框架,为不同像素分配独立的时间步,并重构像素级去噪过程。通过动态调节各像素的去噪时间表,提示相关区域比无关区域更缓慢去噪,从而获得更清晰的像素间上下文信息。最终,这些区域在生成图像中实现更好的文本对齐。大量实验表明,该方法在多样化提示下均显著提升文本到图像的对齐性能。代码已开源:https://github.com/hu-zijing/AsynDM。
原文摘要 · Abstract (English)
Diffusion models have achieved impressive results in generating high-quality images. Yet, they often struggle to faithfully align the generated images with the input prompts. This limitation is associated with synchronous denoising, where all pixels simultaneously evolve from random noise to clear images. As a result, during generation, the prompt-related regions can only reference the unrelated regions at the same noise level, failing to obtain clear context and ultimately impairing text-to-image alignment. To address this issue, we propose asynchronous diffusion models -- a novel framework that allocates distinct timesteps to different pixels and reformulates the pixel-wise denoising process. By dynamically modulating the timestep schedules of individual pixels, prompt-related regions are denoised more gradually than unrelated regions, thereby allowing them to leverage clearer inter-pixel context. Consequently, these prompt-related regions achieve better alignment in the final images. Extensive experiments demonstrate that our asynchronous diffusion models can significantly improve text-to-image alignment across diverse prompts. The code repository for this work is available at https://github.com/hu-zijing/AsynDM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。