arXiv:2505.10999cs.CV2025-05被引 3

让扩散模型自己反馈中间特征,提升生成质量

Conditioning Residuals for Diffusion Models via Representation Feedback

  • 用残差形式将中间特征反馈到条件嵌入,无需额外模块
  • 在多个数据集和模型上均提升生成性能与表征能力
  • 轻量级设计,适合各类扩散模型架构,适配性强

扩散模型已成为多媒体生成的通用基础,其训练过程中会自然产生有用的中间表示。现有架构通常将这些表示沿主特征流传递,未显式将其语义反馈至后续去噪层。而这类主干网络已具备通过预定义输入进行全局调制的条件路径。本文探究该路径是否可同时用于传输内部推断出的、随样本变化的语义信息。提出「条件残差」机制:将聚合特征转化为残差并添加至条件嵌入中。通过反馈紧凑的特征摘要,实现自适应生成引导,并促进更紧致的语义瓶颈,无需外部编码器、辅助目标或采样时修改。支持在UNet和DiT主干中单层或多层反馈,开销极小。跨多种扩散范式、主干配置与数据集的实验表明,生成性能持续提升,下游线性探测与分割任务的表征能力也更强。机制分析揭示了更优的生成训练动态与重构的特征结构,证明了一种从内部增强扩散主干的稳健且通用的方法。

原文摘要 · Abstract (English)

Diffusion models now serve as a common foundation for multimedia generation, and useful intermediate representations emerge during their generative training. Standard architectures, however, propagate these representations through the main feature stream, without explicitly reintroducing their encoded semantics to later denoising layers. Meanwhile, such backbones already provide a conditioning pathway for global modulation by predefined inputs. This work examines whether this native pathway can also route internally inferred semantics as evolving, sample-dependent cues. We propose Conditioning Residuals, a lightweight feedback mechanism that converts aggregated features into residuals added to condition embeddings. By feeding back compact feature summaries, it provides adaptive generative guidance and encourages a tighter semantic bottleneck, without external encoders, auxiliary objectives, or sampling-time changes. It supports feedback at one or multiple depths in UNet and DiT backbones, with negligible overhead. Across diffusion formulations, backbone configurations, and datasets, experiments show consistent gains in generative performance, along with stronger representations in downstream linear probing and segmentation. Mechanistic analyses reveal improved generative training dynamics and reshaped feature structure, suggesting a grounded, generalizable way to enhance diffusion backbones from within.

扩散模型特征反馈生成质量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。