让图像不同区域按各自节奏去噪,生成更灵活真实。
AsyncPatch Diffusion: spatially-flexible image generation

- 给图像不同区域分配独立噪声水平,实现异步去噪
- 在ImageNet和LSUN上达到与传统模型相当的生成质量
- 无需微调即可直接用于修复,适合需要局部控制的场景
标准扩散模型对整个样本使用单一噪声水平,迫使所有空间区域遵循相同去噪路径。本文提出AsyncPatch Diffusion,一种联合扩散框架,为图像像素或潜在令牌等不同输入维度分配不同的噪声水平。我们证明该异步污染定义了有效的生成过程,并首次建立了该过程的合法变分下界(ELBO)。结果显示,单个预训练模型即可实现空间自适应生成,不同区域按不同调度去噪。训练中关键挑战是:独立采样噪声水平会过度强调高度异构配置,而忽视采样时至关重要的同质噪声水平。为此,我们设计了一种受控噪声水平采样器,调节平均污染程度及其空间变异性。AsyncPatch在ImageNet 256和LSUN上生成质量与传统扩散模型相当,且天然支持修复任务,无需特定微调。我们进一步引入输入引导机制,利用干净或部分污染区域指导未知区域生成,提升局部一致性与纹理匹配。最后,我们展示了不确定性引导加速和自回归采样等自适应生成策略。
原文摘要 · Abstract (English)
Standard diffusion models corrupt an entire sample with a single shared noise level, forcing all spatial regions to follow the same denoising trajectory. We introduce AsyncPatch Diffusion, a joint-diffusion framework that assigns distinct noise levels to different input dimensions, such as image pixels, or latent tokens. We show how this asynchronous corruption defines a valid generative process while supporting a richer family of spatially heterogeneous denoising trajectories, and prove the first valid ELBO for this process. We show that a single pretrained model can perform spatially adaptive generation, where different regions are denoised on different schedules. A key challenge is training: naive independent noise-level sampling overemphasizes highly heterogeneous configurations and underrepresents homogeneous noise levels, that are crucial during sampling. We address this with a controlled noise-level sampler that regulates both the average corruption level and its spatial variability. AsyncPatch achieves generation quality comparable to conventional diffusion on ImageNet 256 and LSUN, while being natively suited for inpainting without task-specific fine-tuning. We further introduce input guidance, which uses clean or partially corrupted regions to guide the generation of unknown regions, improving local consistency and texture matching. Finally, we demonstrate adaptive generation strategies including uncertainty-guided acceleration and autoregressive sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。