无需训练即可提升图像生成分辨率，速度快质量高
ScaleDiff: Higher-Resolution Image Synthesis via Efficient and Model-Agnostic Diffusion
- 用非重叠块注意力机制减少计算冗余，提升效率
- 在U-Net与Diffusion Transformer上均达当前最优性能
- 适合需要快速扩展分辨率的图像生成应用
文本到图像扩散模型在生成超过训练分辨率的图像时性能下降。现有无训练方法或计算开销大，或不兼容最新扩散Transformer模型。本文提出ScaleDiff，一种无需训练、模型无关的高效率框架，可扩展预训练扩散模型的分辨率。核心是局部块注意力（NPA），通过非重叠块减少自注意力层冗余计算；集成至SDEdit流程中，并引入潜在频率混合（LFM）以更好生成细节；同时采用结构引导增强去噪过程中的全局结构。实验表明，ScaleDiff在图像质量和推理速度上均优于现有无训练方法，在U-Net与Diffusion Transformer架构上表现最佳。
原文摘要 · Abstract (English)
Text-to-image diffusion models often exhibit degraded performance when generating images beyond their training resolution. Recent training-free methods can mitigate this limitation, but they often require substantial computation or are incompatible with recent Diffusion Transformer models. In this paper, we propose ScaleDiff, a model-agnostic and highly efficient framework for extending the resolution of pretrained diffusion models without any additional training. A core component of our framework is Neighborhood Patch Attention (NPA), an efficient mechanism that reduces computational redundancy in the self-attention layer with non-overlapping patches. We integrate NPA into an SDEdit pipeline and introduce Latent Frequency Mixing (LFM) to better generate fine details. Furthermore, we apply Structure Guidance to enhance global structure during the denoising process. Experimental results demonstrate that ScaleDiff achieves state-of-the-art performance among training-free methods in terms of both image quality and inference speed on both U-Net and Diffusion Transformer architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。