arXiv:2504.01515cs.CVcs.AI2025-04CVPR被引 10

无需训练即可灵活组合多种条件,实现精准图像生成。

Training-free Dense-Aligned Diffusion Guidance for Modular Conditional Image Synthesis

  • 将条件拆分为文本、布局、拖拽三类单元,分别设计对齐模块。
  • 在文本/布局/拖拽条件下生成效果优于现有方法,支持多条件组合。
  • 适合需要快速适配新条件的图像生成应用,如设计工具或虚拟场景构建。

条件图像生成在艺术创作、虚拟现实等领域具有广泛应用。然而,现有生成方法通常任务导向、适用范围有限,仅支持单一条件且泛化能力差。本文提出一种新型方法,将条件图像生成视为多种基础条件单元的模块化组合。具体地,将条件划分为文本、布局和拖拽三类核心单元,并为每类设计专用对齐模块:针对文本条件,引入密集概念对齐(DCA)模块,通过融合多样化文本概念实现像素级图文对齐;针对布局条件,提出密集几何对齐(DGA)模块,强化几何约束以保持空间结构;针对拖拽条件,设计密集运动对齐(DMA)模块,施加多层次运动正则化,确保每个像素沿预期轨迹移动且无视觉伪影。通过灵活插入与组合这些模块,框架显著提升模型对多样条件生成任务的适应性,大幅扩展应用场景。大量实验表明,该框架在文本描述、语义掩码(边界框)、拖拽操作及其组合等多种条件下均表现优异。代码已公开于 https://github.com/ZixuanWang0525/DADG。

原文摘要 · Abstract (English)

Conditional image synthesis is a crucial task with broad applications, such as artistic creation and virtual reality. However, current generative methods are often task-oriented with a narrow scope, handling a restricted condition with constrained applicability. In this paper, we propose a novel approach that treats conditional image synthesis as the modular combination of diverse fundamental condition units. Specifically, we divide conditions into three primary units: text, layout, and drag. To enable effective control over these conditions, we design a dedicated alignment module for each. For the text condition, we introduce a Dense Concept Alignment (DCA) module, which achieves dense visual-text alignment by drawing on diverse textual concepts. For the layout condition, we propose a Dense Geometry Alignment (DGA) module to enforce comprehensive geometric constraints that preserve the spatial configuration. For the drag condition, we introduce a Dense Motion Alignment (DMA) module to apply multi-level motion regularization, ensuring that each pixel follows its desired trajectory without visual artifacts. By flexibly inserting and combining these alignment modules, our framework enhances the model's adaptability to diverse conditional generation tasks and greatly expands its application range. Extensive experiments demonstrate the superior performance of our framework across a variety of conditions, including textual description, segmentation mask (bounding box), drag manipulation, and their combinations. Code is available at https://github.com/ZixuanWang0525/DADG.

图像生成扩散模型条件控制模块化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。