动态重加权U-Net注意力块，提升图像生成质量与效率
Dynamic Importance in Diffusion U-Net for Enhanced Image Synthesis
- 通过理论证明注意力块输出重加权可免费提升信噪比
- 设计重要性探测器，量化注意力块在去噪过程中的动态变化
- 自适应重加权调度提升生成图像的美学与身份一致性
传统扩散模型通常采用U-Net架构。先前研究揭示了U-Net中注意力模块的作用，但忽略了其在推理过程中重要性的动态演变，限制了进一步优化图像应用的潜力。本研究首次从理论上证明，在U-Net的Transformer模块输出上进行重加权，可在采样过程中免费提升信噪比。接着提出Importance Probe，用于揭示并量化变压器块在整个去噪过程中的重要性动态变化。最后，设计了一种针对特定图像生成与编辑任务的自适应重要性重加权策略。实验表明，该方法显著提升了推理效率，并增强了样本的美学质量与身份一致性。该方法可无缝集成至任何基于U-Net的架构中。
原文摘要 · Abstract (English)
Traditional diffusion models typically employ a U-Net architecture. Previous studies have unveiled the roles of attention blocks in the U-Net. However, they overlook the dynamic evolution of their importance during the inference process, which hinders their further exploitation to improve image applications. In this study, we first theoretically proved that, re-weighting the outputs of the Transformer blocks within the U-Net is a "free lunch" for improving the signal-to-noise ratio during the sampling process. Next, we proposed Importance Probe to uncover and quantify the dynamic shifts in importance of the Transformer blocks throughout the denoising process. Finally, we design an adaptive importance-based re-weighting schedule tailored to specific image generation and editing tasks. Experimental results demonstrate that, our approach significantly improves the efficiency of the inference process, and enhances the aesthetic quality of the samples with identity consistency. Our method can be seamlessly integrated into any U-Net-based architecture. Code: https://github.com/Hytidel/UNetReweighting
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。