arXiv:2410.09306cs.CV2024-10

通过像素级时间感知机制,加速扩散修复模型采样速度。

TD-Paint: Faster Diffusion Inpainting Through Time Aware Pixel Conditioning

  • 在扩散过程早期嵌入已知像素信息,引导生成轨迹
  • 采样速度提升显著,图像质量保持不变
  • 无需修改网络结构,适合快速部署

扩散模型在图像修复任务中表现优异,但采样速度慢限制了实际应用。我们发现现有模型在初始采样阶段对输入条件利用不足,导致生成轨迹偏离数据流形,需复杂同步机制修正。为此提出时间感知扩散修复方法(TD-Paint),通过建模像素级可变噪声水平,使模型从一开始就高效利用已知像素值,引导生成过程贴近目标流形。该方法在不改变网络结构的前提下,显著加速采样过程。跨三个数据集的实验表明,TD-Paint在保持低复杂度的同时,优于当前最先进的扩散修复模型。

原文摘要 · Abstract (English)

Diffusion models have emerged as highly effective techniques for inpainting, however, they remain constrained by slow sampling rates. While recent advances have enhanced generation quality, they have also increased sampling time, thereby limiting scalability in real-world applications. We investigate the generative sampling process of diffusion-based inpainting models and observe that these models make minimal use of the input condition during the initial sampling steps. As a result, the sampling trajectory deviates from the data manifold, requiring complex synchronization mechanisms to realign the generation process. To address this, we propose Time-aware Diffusion Paint (TD-Paint), a novel approach that adapts the diffusion process by modeling variable noise levels at the pixel level. This technique allows the model to efficiently use known pixel values from the start, guiding the generation process toward the target manifold. By embedding this information early in the diffusion process, TD-Paint significantly accelerates sampling without compromising image quality. Unlike conventional diffusion-based inpainting models, which require a dedicated architecture or an expensive generation loop, TD-Paint achieves faster sampling times without architectural modifications. Experimental results across three datasets show that TD-Paint outperforms state-of-the-art diffusion models while maintaining lower complexity.

图像修复扩散模型加速采样

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。