用3D高斯点云和扩散模型,让机器人从一张图找目标物。
SplatSearch: Instance Image Goal Navigation for Mobile Robots using 3D Gaussian Splatting and Diffusion Models
- 用稀疏重建的3D高斯点云生成多视角图像。
- 扩散模型补全缺失区域,提升与目标图的匹配鲁棒性。
- 结合语义与视觉信息,智能选择搜索路径,适合真实场景导航。
实例图像目标导航(IIN)问题要求移动机器人在未知环境中,仅凭目标物体或人物的一张参考图像,搜索特定目标。该问题在参考图像视角任意、且机器人依赖稀疏视图场景重建时尤为困难。本文提出SplatSearch,一种利用稀疏在线3D高斯点云(3DGS)重建的新架构。SplatSearch基于稀疏3DGS地图渲染候选物体周围多个视角,并使用多视角扩散模型补全图像缺失区域,实现对目标图像的鲁棒特征匹配。同时引入新颖的前沿探索策略,结合合成视角的视觉上下文与目标图像的语义信息,评估前沿位置,优先选择与目标图像在语义和视觉上相关的区域。在逼真家居环境与真实世界场景中的大量实验表明,SplatSearch在成功率与成功路径长度上均优于现有最先进方法。消融实验验证了其设计选择的有效性。
原文摘要 · Abstract (English)
The Instance Image Goal Navigation (IIN) problem requires mobile robots deployed in unknown environments to search for specific objects or people of interest using only a single reference goal image of the target. This problem can be especially challenging when: 1) the reference image is captured from an arbitrary viewpoint, and 2) the robot must operate with sparse-view scene reconstructions. In this paper, we address the IIN problem, by introducing SplatSearch, a novel architecture that leverages sparse-view 3D Gaussian Splatting (3DGS) reconstructions. SplatSearch renders multiple viewpoints around candidate objects using a sparse online 3DGS map and uses a multi-view diffusion model to complete missing regions of the rendered images, enabling robust feature matching against the goal image. A novel frontier exploration policy is introduced which uses visual context from the synthesized viewpoints with semantic context from the goal image to evaluate frontier locations, allowing the robot to prioritize frontiers that are semantically and visually relevant to the goal image. Extensive experiments in photorealistic home and real-world environments validate the higher performance of SplatSearch against current state-of-the-art methods in terms of Success Rate and Success Path Length. An ablation study confirms the design choices of SplatSearch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。