提升扩散模型的条件生成效率，让多模态控制更快速实用。
OminiControl2: Efficient Conditioning for Diffusion Transformers
- 动态压缩条件输入，只保留关键语义标记。
- 条件特征复用，避免重复计算，降低90%以上开销。
- 适合需要高效多条件图像生成的研究与应用。
文本到图像的扩散变换器（DiT）模型的细粒度控制仍是实际部署中的关键挑战。尽管近期如OminiControl等方法已实现多样控制信号的可控生成，但在处理长条件输入时仍存在显著计算效率问题。我们提出OminiControl2，一种高效的图像条件生成框架。该框架引入两项核心创新：(1) 动态压缩策略，在生成过程中仅保留最具语义相关性的标记以简化条件输入；(2) 条件特征复用机制，仅在初始阶段计算条件标记特征，并在去噪步骤中重复使用。这些架构改进在保持原有参数效率和多模态灵活性的同时，大幅降低计算成本。实验表明，相较于前代方法，OminiControl2将条件处理开销降低超过90%，在多条件生成场景下整体提速5.9倍。该效率提升使复杂、多模态控制在高保真图像合成中得以实用化。
原文摘要 · Abstract (English)
Fine-grained control of text-to-image diffusion transformer models (DiT) remains a critical challenge for practical deployment. While recent advances such as OminiControl and others have enabled a controllable generation of diverse control signals, these methods face significant computational inefficiency when handling long conditional inputs. We present OminiControl2, an efficient framework that achieves efficient image-conditional image generation. OminiControl2 introduces two key innovations: (1) a dynamic compression strategy that streamlines conditional inputs by preserving only the most semantically relevant tokens during generation, and (2) a conditional feature reuse mechanism that computes condition token features only once and reuses them across denoising steps. These architectural improvements preserve the original framework's parameter efficiency and multi-modal versatility while dramatically reducing computational costs. Our experiments demonstrate that OminiControl2 reduces conditional processing overhead by over 90% compared to its predecessor, achieving an overall 5.9$\times$ speedup in multi-conditional generation scenarios. This efficiency enables the practical implementation of complex, multi-modal control for high-quality image synthesis with DiT models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。