用微分方程解析流匹配修复图像,提升效果与速度
Plug-and-Play Image Restoration with Flow Matching: A Continuous Viewpoint
- 将插件式流匹配转为随机微分方程,建立连续模型
- 量化修复误差并优化步长调度与网络稳定性
- 通过外推加速模型,性能超越现有方法
基于流匹配的生成模型已融入插件式图像修复框架,形成插件式流匹配(PnP-Flow)模型,在图像修复任务中表现出显著的实证效果。然而其理论理解滞后于实践进展。本文推导出PnP-Flow的连续极限,构建了其随机微分方程(SDE)代理模型。该模型揭示两个关键洞见:(1)可量化图像修复误差,指导优化步长调度并正则化神经网络参数化向量场的Lipschitz常数以降低误差;(2)启发通过外推加速现成的PnP-Flow模型,得到所提SDE模型的缩放版本。在图像去噪、去模糊、超分辨率和补全等基准任务上验证了改进后的PnP-Flow有效性。数值结果表明,本方法显著优于基线PnP-Flow及其他先进方法,在各项评估指标上均取得更优表现。
原文摘要 · Abstract (English)
Flow matching-based generative models have been integrated into the plug-and-play image restoration framework, and the resulting plug-and-play flow matching (PnP-Flow) model has achieved some remarkable empirical success for image restoration. However, the theoretical understanding of PnP-Flow lags its empirical success. In this paper, we derive a continuous limit for PnP-Flow, resulting in a stochastic differential equation (SDE) surrogate model of PnP-Flow. The SDE model provides two particular insights to improve PnP-Flow for image restoration: (1) It enables us to quantify the error for image restoration, informing us to improve step scheduling and regularize the Lipschitz constant of the neural network-parameterized vector field for error reduction. (2) It informs us to accelerate off-the-shelf PnP-Flow models via extrapolation, resulting in a rescaled version of the proposed SDE model. We validate the efficacy of the SDE-informed improved PnP-Flow using several benchmark tasks, including image denoising, deblurring, super-resolution, and inpainting. Numerical results show that our method significantly outperforms the baseline PnP-Flow and other state-of-the-art approaches, achieving superior performance across evaluation metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。