UNITY让扩散模型高效支持多种条件生成，且推理更快更省内存。
UNITY: Attention Flow Networks for Adaptive Conditioning in Diffusion
- 用两阶段训练统一学习多条件特征，再针对性优化
- 在多个数据集上达到顶尖图像质量，内存消耗更低
- 适合需要快速部署多条件生成的开发者使用
我们提出 UNITY，一种面向基于扩散模型图像生成的通用到专用适配器，实现高效且可扩展的复合条件生成。与以往为每种条件模态单独训练适配器的方法不同，UNITY 在不修改底层架构的前提下，联合学习多种条件类型间的共享语义，并实现后续专业化。其两阶段训练范式包含：第一阶段（占总训练步数一半）的通用阶段，用于捕捉所有条件模态间的跨模态表示；第二阶段（剩余训练预算）的专化阶段，用于精炼特定模态特征。UNITY 核心由可变形注意力流（MAF）网络和形态包装器模块构成，通过可学习的流场与基于注意力的融合，实现通道感知与空间自适应特征对齐。该恒定复杂度设计支持单条件与复合条件场景下的灵活运行，显著降低推理延迟与内存占用。在多个数据集上的大量实验表明，UNITY 在保持卓越内存效率的同时，实现了当前最优的图像保真度。
原文摘要 · Abstract (English)
We introduce UNITY, a Universal-to-Specialized adapter for efficient and scalable composite conditioning in diffusion based image generation. Unlike prior methods that train separate adapters for each conditioning modality, UNITY jointly learns shared semantics across multiple conditioning types and subsequently specializes without modifying the underlying architecture. The proposed two stage training paradigm consists of a Universal Stage that captures cross modal representations across all conditioning modalities using half of the total training steps, followed by a Specialization Stage that refines modality specific features using the remaining training budget. At the core of UNITY are the Morphable Attention Flow (MAF) Network and Morph Wrapper modules, which enable channel aware and spatially adaptive feature alignment through learnable flow fields and attention based fusion. This constant complexity formulation supports flexible operation under both single and composite conditioning settings while significantly reducing inference latency and memory consumption. Extensive experiments across multiple datasets demonstrate that UNITY achieves state of the art image fidelity while maintaining superior memory efficiency. Code: https://github.com/arya-domain/UNITY
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。