让机器人通过动态更新物体与容器关系图,精准找到移动过的日常物品。
OpenObject-NAV: Open-Vocabulary Object-Oriented Navigation Based on Dynamic Carrier-Relationship Scene Graph
- 构建开放词汇的载体-关系场景图,实时追踪物体与容器的位置变化。
- 在Habitat模拟器中实现对移动目标的高效定位,成功率显著提升。
- 结合大模型常识与视觉语言相似性,适合真实机器人部署应用。
日常生活中,杯子等常用物品位置不固定且同类有多件,其承载容器也常变动,导致机器人难以高效导航至特定实例。现有方法多聚焦语义层面,缺乏动态更新场景表征的能力。本文捕捉常用物品与其静态载体之间的关系,构建开放词汇的载体-关系场景图(CRSG),并在导航过程中动态更新携带状态以反映场景变化。基于该图,提出将导航过程建模为马尔可夫决策过程的实例导航策略,每一步决策结合大语言模型的常识知识与视觉-语言特征相似性。我们在Habitat模拟器中设计了一系列长序列导航任务,针对常见日常物品验证了算法有效性。结果表明,通过持续更新CRSG,机器人能高效定位已移动的目标。此外,算法已在真实机器人上部署并验证了实用性。
原文摘要 · Abstract (English)
In everyday life, frequently used objects like cups often have unfixed positions and multiple instances within the same category, and their carriers frequently change as well. As a result, it becomes challenging for a robot to efficiently navigate to a specific instance. To tackle this challenge, the robot must capture and update scene changes and plans continuously. However, current object navigation approaches primarily focus on semantic-level and lack the ability to dynamically update scene representation. This paper captures the relationships between frequently used objects and their static carriers. It constructs an open-vocabulary Carrier-Relationship Scene Graph (CRSG) and updates the carrying status during robot navigation to reflect the dynamic changes of the scene. Based on the CRSG, we further propose an instance navigation strategy that models the navigation process as a Markov Decision Process. At each step, decisions are informed by Large Language Model's commonsense knowledge and visual-language feature similarity. We designed a series of long-sequence navigation tasks for frequently used everyday items in the Habitat simulator. The results demonstrate that by updating the CRSG, the robot can efficiently navigate to moved targets. Additionally, we deployed our algorithm on a real robot and validated its practical effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。