用多张照片联合去噪，提升图像恢复效果。
SIR-DIFF: Sparse Image Sets Restoration with Multi-View Diffusion Model
- 利用多视角图像间的互补信息，联合重建清晰图像。
- 在去模糊和超分辨率任务上优于单图及视频方法。
- 输出三维一致图像，适合3D重建等多视角应用。
计算机视觉领域已发展出多种从单张退化照片中恢复真实场景信息的技术，但该任务极为病态。本文提出一种新思路：联合处理同一场景的多张退化照片进行去噪。核心假设是，这些图像包含互补信息，联合使用可更好约束恢复问题。为此，我们构建了一个强大的多视图扩散模型，通过挖掘多视角关系，联合生成无噪声图像。实验表明，该方法在图像去模糊与超分辨率任务上超越现有单图及视频基线方法。关键的是，模型训练目标为生成三维一致图像，使其成为3D重建或姿态估计等需要鲁棒多视角融合任务的有力工具。
原文摘要 · Abstract (English)
The computer vision community has developed numerous techniques for digitally restoring true scene information from single-view degraded photographs, an important yet extremely ill-posed task. In this work, we tackle image restoration from a different perspective by jointly denoising multiple photographs of the same scene. Our core hypothesis is that degraded images capturing a shared scene contain complementary information that, when combined, better constrains the restoration problem. To this end, we implement a powerful multi-view diffusion model that jointly generates uncorrupted views by extracting rich information from multi-view relationships. Our experiments show that our multi-view approach outperforms existing single-view image and even video-based methods on image deblurring and super-resolution tasks. Critically, our model is trained to output 3D consistent images, making it a promising tool for applications requiring robust multi-view integration, such as 3D reconstruction or pose estimation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。