用双像素传感器自动解决3D重建的尺度模糊问题
DP-SfM: Dual-Pixel Structure-from-Motion without Scale Ambiguity
- 利用双像素图像的散焦模糊信息推断绝对尺度
- 无需参考物或标定，可恢复真实尺寸的3D结构
- 适合无标尺环境下的便携式3D重建应用
多视角3D重建（即运动恢复结构后接多视图立体）是计算机视觉的基础。通常，若场景中无已知尺寸的参考物，则重建结果存在未知尺度模糊。本文提出，使用双像素（DP）传感器拍摄的多视角图像可自动消除尺度模糊，无需参考物或预先标定。关键在于，双像素图像中的散焦模糊提供了足够信息，结合多视角重建获得的相对深度图（未定尺度），即可确定绝对尺度。基于此，我们设计了一种简单有效的线性方法估计真实尺度，并通过基于强度的优化阶段，利用跨视图模糊核将左右图像向彼此回移，实现对齐。实验表明，该方法在不同相机和镜头拍摄的多样化场景中均有效。代码与数据见：https://github.com/lilika-makabe/dp-sfm-tpami.git
原文摘要 · Abstract (English)
Multi-view 3D reconstruction, namely, structure-from-motion followed by multi-view stereo, is a fundamental component of 3D computer vision. In general, multi-view 3D reconstruction suffers from an unknown scale ambiguity unless a reference object of known size is present in the scene. In this article, we show that multi-view images captured using a dual-pixel (DP) sensor can automatically resolve the scale ambiguity, without requiring a reference object or prior calibration. Specifically, the defocus blur observed in DP images provides sufficient information to determine the absolute scale when paired with depth maps (up to scale) recovered from multi-view 3D reconstruction. Based on this observation, we develop a simple yet effective linear method to estimate the absolute scale, followed by the intensity-based optimization stage that aligns the left and right DP images by shifting them back toward each other using cross-view blur kernels. Experiments demonstrate the effectiveness of the proposed approach across diverse scenes captured with different cameras and lenses. Code and data are available at https://github.com/lilika-makabe/dp-sfm-tpami.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。