arXiv:2410.02423cs.CVcs.LG2024-10ICLR被引 99

将流匹配模型融入插件式恢复框架,提升图像修复效率与效果。

PnP-Flow: Plug-and-Play Image Restoration with Flow Matching

  • 用预训练流匹配模型构建时变去噪器,结合优化迭代求解逆问题
  • 在去噪、超分辨率、去模糊和修复任务上均超越现有方法
  • 无需反向传播或迹计算,计算高效且内存友好,适合实际部署

本文提出插件式流匹配(PnP-Flow),用于解决成像逆问题。传统插件式方法利用预训练去噪器(如深度神经网络)在优化中提升性能,但在生成类任务(如图像修复)中存在局限;而流匹配虽在图像采样上表现优异,却缺乏高效的图像修复应用方式。我们通过预训练的流匹配模型定义时变去噪器,将该方法嵌入插件式框架,交替执行数据保真梯度下降、投影到学习的流路径以及去噪操作。该方法避免了对常微分方程的反向传播和迹计算,具有计算高效、内存友好的优势。我们在去噪、超分辨率、去模糊和图像修复任务上进行评估,结果表明其性能优于现有的插件式算法及基于流匹配的先进方法。

原文摘要 · Abstract (English)

In this paper, we introduce Plug-and-Play (PnP) Flow Matching, an algorithm for solving imaging inverse problems. PnP methods leverage the strength of pre-trained denoisers, often deep neural networks, by integrating them in optimization schemes. While they achieve state-of-the-art performance on various inverse problems in imaging, PnP approaches face inherent limitations on more generative tasks like inpainting. On the other hand, generative models such as Flow Matching pushed the boundary in image sampling yet lack a clear method for efficient use in image restoration. We propose to combine the PnP framework with Flow Matching (FM) by defining a time-dependent denoiser using a pre-trained FM model. Our algorithm alternates between gradient descent steps on the data-fidelity term, reprojections onto the learned FM path, and denoising. Notably, our method is computationally efficient and memory-friendly, as it avoids backpropagation through ODEs and trace computations. We evaluate its performance on denoising, super-resolution, deblurring, and inpainting tasks, demonstrating superior results compared to existing PnP algorithms and Flow Matching based state-of-the-art methods.

图像修复流匹配插件式方法去噪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。