将物理测量模型融入扩散模型,实现高效通用的图像逆问题求解
Diff-Unfolding: A Model-Based Score Learning Framework for Inverse Problems
- 通过展开优化流程,将测量模型与图像先验解耦训练
- 在图像修复和加速MRI中提升性能,最高增益2dB PSNR
- 模型轻量高效,推理仅需0.63秒,可直接替换算子复用
扩散模型广泛用于图像逆问题中的先验建模。本文提出 extit{Diff-Unfolding},一种基于物理测量算子的条件扩散模型后验得分函数学习框架。该方法将后验得分学习形式化为可展开的优化过程,使测量模型与学习到的图像先验解耦。此设计允许在推理时仅更换前向算子即可跨逆问题泛化,无需重新训练。理论证明了该展开方法等价于基于模型的复合优化形式。在图像修复和加速MRI任务上的大量实验表明,Diff-Unfolding达到当前最优性能,最高提升PSNR达2 dB,LPIPS降低22.7%,且模型紧凑(47M参数),推理高效(256×256图像仅需0.72秒)。经优化的C++/LibTorch实现进一步将推理时间缩短至0.63秒,凸显其实际应用价值。
原文摘要 · Abstract (English)
Diffusion models are extensively used for modeling image priors for inverse problems. We introduce \emph{Diff-Unfolding}, a principled framework for learning posterior score functions of \emph{conditional diffusion models} by explicitly incorporating the physical measurement operator into a modular network architecture. Diff-Unfolding formulates posterior score learning as the training of an unrolled optimization scheme, where the measurement model is decoupled from the learned image prior. This design allows our method to generalize across inverse problems at inference time by simply replacing the forward operator without retraining. We theoretically justify our unrolling approach by showing that the posterior score can be derived from a composite model-based optimization formulation. Extensive experiments on image restoration and accelerated MRI show that Diff-Unfolding achieves state-of-the-art performance, improving PSNR by up to 2 dB and reducing LPIPS by $22.7\%$, while being both compact (47M parameters) and efficient (0.72 seconds per $256 \times 256$ image). An optimized C++/LibTorch implementation further reduces inference time to 0.63 seconds, underscoring the practicality of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。