用单目飞行时间相机实现快速高精度动态3D重建
Time of the Flight of the Gaussians: Optimizing Depth Indirectly in Dynamic Radiance Fields
- 基于3D高斯泼溅，利用原始传感器数据间接优化深度
- 在受限条件下重建准确率媲美神经体积方法，速度提升100倍
- 适合快速运动场景，如挥动棒球棒的动态重建
我们提出一种方法，仅用单目连续波飞行时间（C-ToF）相机的原始传感器采样，即可实现动态场景的高保真3D重建，精度与神经体积方法相当甚至更优，且速度提升100倍。从单一视角快速实现高质量动态3D重建是计算机视觉中的重大挑战。在C-ToF辐射场重建中，目标属性——深度——并非直接测量，带来额外困难。该问题在使用快速基础结构表示（如3D高斯泼溅）时影响显著，此类方法在多视角数据下表现良好，但单视角优化时极易失效。本文在优化中引入两项启发式策略，显著提升高斯表示的几何精度。实验表明，本方法在受限的C-ToF感知条件下仍能生成准确重建结果，包括快速运动如挥动棒球棒的场景。
原文摘要 · Abstract (English)
We present a method to reconstruct dynamic scenes from monocular continuous-wave time-of-flight (C-ToF) cameras using raw sensor samples that achieves similar or better accuracy than neural volumetric approaches and is 100x faster. Quickly achieving high-fidelity dynamic 3D reconstruction from a single viewpoint is a significant challenge in computer vision. In C-ToF radiance field reconstruction, the property of interest-depth-is not directly measured, causing an additional challenge. This problem has a large and underappreciated impact upon the optimization when using a fast primitive-based scene representation like 3D Gaussian splatting, which is commonly used with multi-view data to produce satisfactory results and is brittle in its optimization otherwise. We incorporate two heuristics into the optimization to improve the accuracy of scene geometry represented by Gaussians. Experimental results show that our approach produces accurate reconstructions under constrained C-ToF sensing conditions, including for fast motions like swinging baseball bats. https://visual.cs.brown.edu/gftorf
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。