用大模型规划探索路径，结合信息增益优化地图构建。
Multimodal LLM Guided Exploration and Active Mapping using Fisher Information
- 大模型驱动长程语义规划，3D高斯点云支持高质量视角生成。
- 兼顾环境信息增益与定位误差成本，实现不确定性感知路径选择。
- 在Gibson和Habitat-Matterport数据集上达到当前最优表现。
我们提出一种主动地图构建系统，利用3D高斯点云（3DGS）表示，同时规划长期探索目标与短期动作。现有方法或未利用多模态大语言模型（LLM）的最新进展，或忽视了具身智能体中的定位不确定性问题。本文采用多模态LLM作为零样本长程探索规划器，从语义层面生成探索目标；同时引入不确定性感知的路径提议与选择算法，在最大化环境信息增益的同时最小化定位误差代价。基于3DGS提供的高质量视图合成能力，该方法在Gibson与Habitat-Matterport 3D数据集上实现了当前最优性能。
原文摘要 · Abstract (English)
We present an active mapping system that plans for both long-horizon exploration goals and short-term actions using a 3D Gaussian Splatting (3DGS) representation. Existing methods either do not take advantage of recent developments in multimodal Large Language Models (LLM) or do not consider challenges in localization uncertainty, which is critical in embodied agents. We propose employing multimodal LLMs for long-horizon planning in conjunction with detailed motion planning using our information-based objective. By leveraging high-quality view synthesis from our 3DGS representation, our method employs a multimodal LLM as a zero-shot planner for long-horizon exploration goals from the semantic perspective. We also introduce an uncertainty-aware path proposal and selection algorithm that balances the dual objectives of maximizing the information gain for the environment while minimizing the cost of localization errors. Experiments conducted on the Gibson and Habitat-Matterport 3D datasets demonstrate state-of-the-art results of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。