无需训练,1秒生成百万像素图像,还原参考图结构。
Efficient and Training-Free Single-Image Diffusion Models

- 用多尺度图像块构建数据集,直接计算去噪函数
- 生成质量超越已有训练模型,支持多样应用
- 适配潜在空间,加速至秒级生成,适合实时场景
我们研究如何生成内部结构(由多尺度图像块分布定义)与单张参考图像一致的图像。现有方法虽在单图设置下训练扩散模型,但仍需数小时优化,计算成本高。本文改用参考图像在不同尺度下的图像块构成有限数据集,因块维度小且数量有限,可直接通过闭式最优去噪器高效计算噪声块的得分函数,无需神经网络训练。将该基于块的去噪器融入高效、无训练的图像扩散模型,并揭示其与经典图像修复技术的联系。实验表明,该方法在生成质量与多样性上优于现有训练型单图扩散模型,成功应用于无条件生成、文本引导风格化、图像对称化及重定形。进一步证明其兼容潜在空间扩散,并结合多种加速技术,实现秒级百万像素生成、分钟级十亿像素生成。
原文摘要 · Abstract (English)
We consider the problem of generating images whose internal structure -- defined by the distribution of patches across multiple scales -- matches that of a single reference image. Recent approaches address this problem by training a diffusion model on a single image. But even in this setting, training is computationally expensive and requires hours of optimization. Instead, we model the image using a dataset of its patches at different scales. As this dataset is finite and the dimensionality of its patches is small, the score function for a noisy patch can be computed tractably using an optimal, closed-form denoiser, eliminating the need for neural network training. We integrate this patch-based denoiser into an efficient, training-free image diffusion model, and we describe how our method connects to classical patch-based image restoration techniques. Our approach achieves state-of-the-art generation quality and diversity compared to trained single-image diffusion models, and we demonstrate applications, including unconditional image generation, text-guided stylization, image symmetrization, and retargeting. Further, we show that our approach is compatible with latent space diffusion, and we show multiple additional acceleration techniques to achieve megapixel single-image generation in one second, and gigapixel generation in minutes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。