用信息论选最佳视角,提升3D重建视觉质量与效率
GauSS-MI: Gaussian Splatting Shannon Mutual Information for Active 3D Reconstruction
- 基于香农互信息构建高斯点视觉不确定性度量
- 实测在多种场景下重建质量显著优于现有方法
- 适合需要实时高质量3D重建的研究与应用
本研究解决实时主动视角选择与视觉质量不确定性量化在主动3D重建中的挑战。视觉质量是3D重建的关键。尽管神经辐射场(NeRF)和3D高斯溅射(3DGS)显著提升了重建模型的图像渲染质量,但针对重建需求高效获取输入图像——特别是选择最具信息量的视点——仍是开放问题,对主动重建至关重要。现有研究主要关注几何完整性及未观测区域探索,未直接评估重建模型内的视觉不确定性。为此,本文提出一种概率模型,量化每个高斯点的视觉不确定性。基于香农互信息,我们构建了高斯溅射香农互信息(GauSS-MI)准则,实现从新视角实时评估视觉互信息,支持最优视点选择。GauSS-MI被集成于包含视点与运动规划器的主动重建系统中。在多种模拟与真实场景下的大量实验表明,所提系统在视觉质量和重建效率方面均表现优异。
原文摘要 · Abstract (English)
This research tackles the challenge of real-time active view selection and uncertainty quantification on visual quality for active 3D reconstruction. Visual quality is a critical aspect of 3D reconstruction. Recent advancements such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have notably enhanced the image rendering quality of reconstruction models. Nonetheless, the efficient and effective acquisition of input images for reconstruction-specifically, the selection of the most informative viewpoint-remains an open challenge, which is crucial for active reconstruction. Existing studies have primarily focused on evaluating geometric completeness and exploring unobserved or unknown regions, without direct evaluation of the visual uncertainty within the reconstruction model. To address this gap, this paper introduces a probabilistic model that quantifies visual uncertainty for each Gaussian. Leveraging Shannon Mutual Information, we formulate a criterion, Gaussian Splatting Shannon Mutual Information (GauSS-MI), for real-time assessment of visual mutual information from novel viewpoints, facilitating the selection of next best view. GauSS-MI is implemented within an active reconstruction system integrated with a view and motion planner. Extensive experiments across various simulated and real-world scenes showcase the superior visual quality and reconstruction efficiency performance of the proposed system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。