用手机实现高精度三维重建,让普通用户也能拍出专业级3D模型。
Differentiable Mobile Display Photometric Stereo
- 用手机屏幕显示可学习的光照图案,相机同步拍摄高清图像
- 在3D打印物和落叶数据集上实现厘米级表面法向重建
- 首个支持移动端的可微分光度立体系统,适合移动摄影与科研应用
显示光度立体利用显示器作为可编程光源,通过多样光照条件照亮场景。近期,可微分显示光度立体(DDPS)通过学习显示图案提升了法向重建精度,但受限于固定桌面设置,需偏振相机与台式显示器。本文提出更实用的物理基光度立体方法——可微分移动显示光度立体(DMDPS),利用包含显示屏和相机的移动设备。通过开发移动应用与方法,实现图案同步显示与高质量HDR图像捕获。我们对真实世界3D打印物体进行采集,并通过可微学习过程优化显示图案。在3D打印数据集和首个倒落树叶数据集上验证了DMDPS的有效性。该树叶数据集包含重构的表面法向与反照率,可支持计算机图形学与视觉之外的未来研究。我们认为DMDPS推动了实用化物理基光度立体的发展。
原文摘要 · Abstract (English)
Display photometric stereo uses a display as a programmable light source to illuminate a scene with diverse illumination conditions. Recently, differentiable display photometric stereo (DDPS) demonstrated improved normal reconstruction accuracy by using learned display patterns. However, DDPS faced limitations in practicality, requiring a fixed desktop imaging setup using a polarization camera and a desktop-scale monitor. In this paper, we propose a more practical physics-based photometric stereo, differentiable mobile display photometric stereo (DMDPS), that leverages a mobile phone consisting of a display and a camera. We overcome the limitations of using a mobile device by developing a mobile app and method that simultaneously displays patterns and captures high-quality HDR images. Using this technique, we capture real-world 3D-printed objects and learn display patterns via a differentiable learning process. We demonstrate the effectiveness of DMDPS on both a 3D printed dataset and a first dataset of fallen leaves. The leaf dataset contains reconstructed surface normals and albedos of fallen leaves that may enable future research beyond computer graphics and vision. We believe that DMDPS takes a step forward for practical physics-based photometric stereo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。