arXiv:2412.06163cs.CV2024-12被引 1

无需训练,异步引导实现高速高分辨率图像生成

ASGDiffusion: Parallel High-Resolution Generation with Asynchronous Structure Guidance

  • 用低分辨率噪声加注意力掩码引导去噪,避免重复图案
  • 并行处理图像块,多卡加速使生成速度大幅提升
  • 适合需要快速生成高质量图像的交互式应用

无需训练的高分辨率(HR)图像生成因大模型训练成本高昂而受到广泛关注。现有方法通常先重建整体结构,再细化局部细节,但仍存在生成图像中模式重复的问题。此外,基于扩散模型的高分辨率生成计算开销大,难以满足交互式应用对实时性的要求。为此,本文提出ASGDiffusion方法,利用预训练扩散模型实现异步结构引导的并行高分辨率生成。为解决模式重复问题,该方法在去噪步骤中引入由注意力掩码加权的低分辨率噪声作为结构引导,以保障语义一致性,显著缓解重复现象。为实现并行生成,进一步设计异步计算策略,分别异步计算图像块的噪声和结构引导。通过多GPU并行加速,大幅提升了生成速度并降低了单卡内存占用。大量实验表明,本方法有效解决了模式重复等常见问题,并达到当前最高水平的高分辨率生成性能。

原文摘要 · Abstract (English)

Training-free high-resolution (HR) image generation has garnered significant attention due to the high costs of training large diffusion models. Most existing methods begin by reconstructing the overall structure and then proceed to refine the local details. Despite their advancements, they still face issues with repetitive patterns in HR image generation. Besides, HR generation with diffusion models incurs significant computational costs. Thus, parallel generation is essential for interactive applications. To solve the above limitations, we introduce a novel method named ASGDiffusion for parallel HR generation with Asynchronous Structure Guidance (ASG) using pre-trained diffusion models. To solve the pattern repetition problem of HR image generation, ASGDiffusion leverages the low-resolution (LR) noise weighted by the attention mask as the structure guidance for the denoising step to ensure semantic consistency. The proposed structure guidance can significantly alleviate the pattern repetition problem. To enable parallel generation, we further propose a parallelism strategy, which calculates the patch noises and structure guidance asynchronously. By leveraging multi-GPU parallel acceleration, we significantly accelerate generation speed and reduce memory usage per GPU. Extensive experiments demonstrate that our method effectively and efficiently addresses common issues like pattern repetition and achieves state-of-the-art HR generation.

图像生成扩散模型并行计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。