让多个3D高斯溅射模型对齐并融合,实现机器人协同建模。
PhotoReg: Photometrically Registering 3D Gaussian Splatting Models
- 利用3D基础模型与图像的双向映射关系,实现多视角3D高斯模型对齐。
- 通过深度估计强制尺度一致,再用像素级光照损失迭代优化融合结果。
- 适用于机器人团队协同环境重建,已在四足机器人实测验证。
构建精准的环境表征对智能机器人决策至关重要。近年来,基于3D高斯溅射(3DGS)的逼真环境模型可实时渲染出类人视觉的2D图像,已迅速成为主流。然而关键挑战仍未解决:如何将多个3DGS模型融合为统一、一致的全局模型?本工作提出PhotoReg框架,利用3D基础模型与图像间的双重映射机制,实现多源3DGS模型的光度对齐。由于3DGS通常由单目相机生成,存在任意尺度问题,PhotoReg通过分析各模型内部的深度估计,主动约束尺度一致性;随后以细粒度光度损失迭代优化姿态与结构,最终生成高质量融合模型。我们在标准基准数据集及自建多机器人数据集上进行评估,包括双足四足机器人实测。代码已开源。
原文摘要 · Abstract (English)
Building accurate representations of the environment is critical for intelligent robots to make decisions during deployment. Advances in photorealistic environment models have enabled robots to develop hyper-realistic reconstructions, which can be used to generate images that are intuitive for human inspection. In particular, the recently introduced \ac{3DGS}, which describes the scene with up to millions of primitive ellipsoids, can be rendered in real time. \ac{3DGS} has rapidly gained prominence. However, a critical unsolved problem persists: how can we fuse multiple \ac{3DGS} into a single coherent model? Solving this problem will enable robot teams to jointly build \ac{3DGS} models of their surroundings. A key insight of this work is to leverage the {duality} between photorealistic reconstructions, which render realistic 2D images from 3D structure, and \emph{3D foundation models}, which predict 3D structure from image pairs. To this end, we develop PhotoReg, a framework to register multiple photorealistic \ac{3DGS} models with 3D foundation models. As \ac{3DGS} models are generally built from monocular camera images, they have \emph{arbitrary scale}. To resolve this, PhotoReg actively enforces scale consistency among the different \ac{3DGS} models by considering depth estimates within these models. Then, the alignment is iteratively refined with fine-grained photometric losses to produce high-quality fused \ac{3DGS} models. We rigorously evaluate PhotoReg on both standard benchmark datasets and our custom-collected datasets, including with two quadruped robots. The code is released at \url{ziweny11.github.io/photoreg}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。