无需训练数据，统一修复多种图像退化问题
LD-RPS: Zero-Shot Unified Image Restoration via Latent Diffusion Recurrent Posterior Sampling
- 用预训练扩散模型递归采样后验分布，实现任务无感知修复
- 在多个真实退化数据集上超越现有方法，保持高鲁棒性
- 适合需要快速部署、跨退化类型通用修复的场景
统一图像修复是低层视觉中极具挑战的任务。现有方法或针对特定任务定制设计，限制了在多种退化类型间的泛化能力；或依赖成对训练数据，受封闭集合约束。为此，我们提出一种全新的、无需数据集的统一方法，通过预训练潜空间扩散模型进行递归后验采样。该方法引入多模态理解模型，在任务无感知条件下为生成模型提供语义先验，并采用轻量模块对齐退化输入与扩散模型生成偏好，结合递归精炼实现后验采样。大量实验表明，本方法在多个真实退化数据集上优于当前最先进方法，验证了其有效性和鲁棒性。代码与数据已公开于 https://github.com/AMAP-ML/LD-RPS。
原文摘要 · Abstract (English)
Unified image restoration is a significantly challenging task in low-level vision. Existing methods either make tailored designs for specific tasks, limiting their generalizability across various types of degradation, or rely on training with paired datasets, thereby suffering from closed-set constraints. To address these issues, we propose a novel, dataset-free, and unified approach through recurrent posterior sampling utilizing a pretrained latent diffusion model. Our method incorporates the multimodal understanding model to provide sematic priors for the generative model under a task-blind condition. Furthermore, it utilizes a lightweight module to align the degraded input with the generated preference of the diffusion model, and employs recurrent refinement for posterior sampling. Extensive experiments demonstrate that our method outperforms state-of-the-art methods, validating its effectiveness and robustness. Our code and data are available at https://github.com/AMAP-ML/LD-RPS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。