用大模型推理+动态地图，让机器人在物体移动时仍能精准寻物。
osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning
- 构建语义地图结合大模型推理，主动应对物体位置变化
- 静态物寻物成功率高且路径更短，动态物查询性能显著领先
- 适合真实场景中频繁移动物品的智能导航任务
现有开放词汇机器人映射方法通过预训练视觉-语言特征丰富稠密几何地图，实现高精度细节表达，可引导机器人根据开放词汇语言指令寻找物体。然而，此类方法面临可扩展性问题，更根本的问题是高精度物体映射极易过时——因物体常被移动。本文提出一种面向物体目标导航的映射与导航系统，从头设计即考虑目标物体可能已移动或未被映射的场景。不追求高保真细节，而是聚焦地图提供环境定位与上下文信息，结合大模型的语义先验进行对象位置推理，并采用主动在线策略导航至目标。仿真与真实世界实验表明，本方法在静态物体查询中路径更短、检索成功率更高；在动态或未映射物体查询中，性能远超先前方法。代码与数据集详见：https://github.com/xiexiexiaoxiexie/osmAG-LLM。
原文摘要 · Abstract (English)
Recent open-vocabulary robot mapping methods enrich dense geometric maps with pre-trained visual-language features, achieving a high level of detail and guiding robots to find objects specified by open-vocabulary language queries. While the issue of scalability for such approaches has received some attention, another fundamental problem is that high-detail object mapping quickly becomes outdated, as objects get moved around a lot. In this work, we develop a mapping and navigation system for object-goal navigation that, from the ground up, considers the possibilities that a queried object can have moved, or may not be mapped at all. Instead of striving for high-fidelity mapping detail, we consider that the main purpose of a map is to provide environment grounding and context, which we combine with the semantic priors of LLMs to reason about object locations and deploy an active, online approach to navigate to the objects. Through simulated and real-world experiments we find that our approach tends to have higher retrieval success at shorter path lengths for static objects and by far outperforms prior approaches in cases of dynamic or unmapped object queries. We provide our code and dataset at: https://github.com/xiexiexiaoxiexie/osmAG-LLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。