用单张模糊图实现高精度深度估计,无需清晰参考图。
Depth Estimation Based on 3D Gaussian Splatting Siamese Defocus
- 基于3D高斯点阵与孪生网络,从焦堆中学习模糊程度。
- 仅需一张模糊图像即可预测失焦图和圆模糊圈(CoC)。
- 自监督机制提升真实场景下深度估计效果,适合无清晰图场景。
深度估计是三维几何的基础任务。虽然双目深度可通过三角测量实现,但单目方法需融合全局与局部信息,更具挑战性。基于失焦的深度(DFD)方法利用镜头模型与参数,从模糊图像中恢复深度,已被证明有效。然而,传统方法依赖全聚焦(AIF)图像进行深度估计,在真实应用中几乎无法获取。为此,本文提出一种基于3D高斯点阵与孪生网络的自监督框架。通过学习同一场景在不同焦距下的模糊程度,该框架可从单张模糊图像预测失焦图与圆模糊圈(CoC),并以失焦图为输入,通过DepthNet实现单目深度估计。3D高斯点阵模型使用预测的CoC渲染模糊图像,其与真实模糊图像的差异作为额外监督信号,反馈给孪生失焦自监督网络。该框架在人工合成与真实模糊数据集上均验证有效。定量与可视化实验表明,本方法在实际应用中具有高度有效性。
原文摘要 · Abstract (English)
Depth estimation is a fundamental task in 3D geometry. While stereo depth estimation can be achieved through triangulation methods, it is not as straightforward for monocular methods, which require the integration of global and local information. The Depth from Defocus (DFD) method utilizes camera lens models and parameters to recover depth information from blurred images and has been proven to perform well. However, these methods rely on All-In-Focus (AIF) images for depth estimation, which is nearly impossible to obtain in real-world applications. To address this issue, we propose a self-supervised framework based on 3D Gaussian splatting and Siamese networks. By learning the blur levels at different focal distances of the same scene in the focal stack, the framework predicts the defocus map and Circle of Confusion (CoC) from a single defocused image, using the defocus map as input to DepthNet for monocular depth estimation. The 3D Gaussian splatting model renders defocused images using the predicted CoC, and the differences between these and the real defocused images provide additional supervision signals for the Siamese Defocus self-supervised network. This framework has been validated on both artificially synthesized and real blurred datasets. Subsequent quantitative and visualization experiments demonstrate that our proposed framework is highly effective as a DFD method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。