RAD让图像修复更快更准,像素级噪声调度实现异步生成。
RAD: Region-Aware Diffusion Models for Image Inpainting
- 为每个像素设计不同噪声调度,支持局部异步生成
- 推理速度比当前最优快100倍,且无需额外组件
- 适配主流扩散模型,训练成本低,适合实际部署
扩散模型在图像生成中表现卓越,其在图像修复中的应用潜力巨大。现有方法或劫持预训练扩散模型的反向过程,或将其转化为条件生成框架,但常需嵌套循环或额外组件。本文提出区域感知扩散模型(RAD),通过为每个像素设计不同的噪声调度,实现局部区域异步生成的同时保留全局上下文信息。采用简单反向过程,无需额外组件,使推理速度比当前最优方法快100倍。此外,利用低秩适配(LoRA)可基于其他预训练扩散模型微调RAD,显著降低训练开销。实验表明,RAD在FFHQ、LSUN Bedroom和ImageNet数据集上均达到定性和定量上的最优性能。
原文摘要 · Abstract (English)
Diffusion models have achieved remarkable success in image generation, with applications broadening across various domains. Inpainting is one such application that can benefit significantly from diffusion models. Existing methods either hijack the reverse process of a pretrained diffusion model or cast the problem into a larger framework, \ie, conditioned generation. However, these approaches often require nested loops in the generation process or additional components for conditioning. In this paper, we present region-aware diffusion models (RAD) for inpainting with a simple yet effective reformulation of the vanilla diffusion models. RAD utilizes a different noise schedule for each pixel, which allows local regions to be generated asynchronously while considering the global image context. A plain reverse process requires no additional components, enabling RAD to achieve inference time up to 100 times faster than the state-of-the-art approaches. Moreover, we employ low-rank adaptation (LoRA) to fine-tune RAD based on other pretrained diffusion models, reducing computational burdens in training as well. Experiments demonstrated that RAD provides state-of-the-art results both qualitatively and quantitatively, on the FFHQ, LSUN Bedroom, and ImageNet datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。