用开放场景图让机器人零样本泛化到新环境找物
Open Scene Graphs for Open-World Object-Goal Navigation
- 用开放场景图作为空间记忆，按环境类型组织结构
- 零样本适配新环境，真实机器人实测表现领先
- 适合研究开放世界导航与通用机器人系统的人
如何构建适用于开放世界语义导航的通用机器人系统？例如，在陌生环境中根据自然语言指令寻找目标物体。为解决此问题，我们提出OSG Navigator，一个由基础模型构成的模块化系统，用于开放世界物体目标导航（ObjectNav）。基础模型具备丰富的世界语义知识，但在大规模下难以有效组织和维护空间信息。OSG Navigator的核心是开放场景图表示，作为其空间记忆。该表示通过开放场景图模式（OSG schemas）分层组织空间信息，每种模式描述一类环境的通用结构，可由简单语义标签（如“家”或“超市”）自动生成。这使得OSG Navigator能实现零样本适应新环境类型。我们在模拟和真实世界中使用Fetch和Spot机器人进行实验，结果表明，OSG Navigator在ObjectNav基准上达到当前最优性能，并在多样化目标、环境和机器人形态间实现零样本泛化。
原文摘要 · Abstract (English)
How can we build general-purpose robot systems for open-world semantic navigation, e.g., searching a novel environment for a target object specified in natural language? To tackle this challenge, we introduce OSG Navigator, a modular system composed of foundation models, for open-world Object-Goal Navigation (ObjectNav). Foundation models provide enormous semantic knowledge about the world, but struggle to organise and maintain spatial information effectively at scale. Key to OSG Navigator is the Open Scene Graph representation, which acts as spatial memory for OSG Navigator. It organises spatial information hierarchically using OSG schemas, which are templates, each describing the common structure of a class of environments. OSG schemas can be automatically generated from simple semantic labels of a given environment, e.g., "home" or "supermarket". They enable OSG Navigator to adapt zero-shot to new environment types. We conducted experiments using both Fetch and Spot robots in simulation and in the real world, showing that OSG Navigator achieves state-of-the-art performance on ObjectNav benchmarks and generalises zero-shot over diverse goals, environments, and robot embodiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。