针对多镜头相机图像超分辨率,提出新自监督框架提升细节恢复能力。
Enhanced Self-Supervised Multi-Image Super-Resolution for Camera Array Images
- 结合单图到单图与多图到多图自监督策略,融合互补优势。
- 在合成与真实数据集上均实现更高清晰度与纹理保真度。
- 适合需要高细节还原的多镜头成像系统研发与优化场景。
传统多图像超分辨率(MISR)方法如快照和视频超分辨率依赖单一相机的连续帧,面临复杂退化与严重遮挡问题,增加图像修复难度。相比之下,多光圈相机阵列通过空间分布视图与采样偏移形成稳定的类盘状分布,提升了观测数据的非冗余性。现有MISR算法未能充分利用这一特性。监督式MISR易过拟合训练数据中的退化模式,而当前自监督学习技术难以恢复细粒度细节。本文深入研究了多图到单图(Multi-to-Single)与多图到多图(Multi-to-Multi)自监督学习方法的优势、局限及适用边界。提出多图到单图引导的多图到多图自监督学习框架,融合两者优点,生成视觉吸引且高保真的含丰富纹理细节图像。该框架为深度神经网络与经典物理驱动变分方法的结合提供了新范式。为进一步提升网络从混叠伪影中恢复高频细节的能力,本文提出一种适用于自监督学习的双变换器相机阵列超分辨率网络。在合成与真实世界数据集上的实验表明所提方法具有优越性能。
原文摘要 · Abstract (English)
Conventional multi-image super-resolution (MISR) methods, such as burst and video SR, rely on sequential frames from a single camera. Consequently, they suffer from complex image degradation and severe occlusion, increasing the difficulty of accurate image restoration. In contrast, multi-aperture camera-array imaging captures spatially distributed views with sampling offsets forming a stable disk-like distribution, which enhances the non-redundancy of observed data. Existing MISR algorithms fail to fully exploit these unique properties. Supervised MISR methods tend to overfit the degradation patterns in training data, and current self-supervised learning (SSL) techniques struggle to recover fine-grained details. To address these issues, this paper thoroughly investigates the strengths, limitations and applicability boundaries of multi-image-to-single-image (Multi-to-Single) and multi-image-to-multi-image (Multi-to-Multi) SSL methods. We propose the Multi-to-Single-Guided Multi-to-Multi SSL framework that combines the advantages of Multi-to-Single and Multi-to-Multi to generate visually appealing and high-fidelity images rich in texture details. The Multi-to-Single-Guided Multi-to-Multi SSL framework provides a new paradigm for integrating deep neural network with classical physics-based variational methods. To enhance the ability of MISR network to recover high-frequency details from aliased artifacts, this paper proposes a novel camera-array SR network called dual Transformer suitable for SSL. Experiments on synthetic and real-world datasets demonstrate the superiority of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。