arXiv:2602.16356cs.ROcs.AI2026-02被引 6

构建可预测物体运动的3D场景图,让机器人在真实环境里更聪明地操作物品。

Articulated 3D Scene Graphs for Open-World Mobile Manipulation

  • 通过点轨迹追踪与统一旋转变换估计,同步推断物体运动与关节参数。
  • 在600次交互中实现92.3%的关节类型识别准确率,显著优于现有方法。
  • 适合做家庭机器人、移动操作任务的开发者和研究者使用。

语义信息已推动3D场景理解与基于功能的物体交互,但现实环境中机器人无法预判物体运动。长时序移动操作需弥合语义、几何与运动学之间的差距。本文提出MoMa-SG框架,用于构建包含大量可操作物体的可动场景的语义-运动3D场景图。给定含多个物体运动的RGB-D序列,通过时间分割物体交互并利用抗遮挡点追踪推断运动轨迹,将点轨迹提升至3D后,采用新颖的统一旋转变换估计法,在单次优化中鲁棒估计转动与移动关节参数。随后通过父-子关系推理,关联物体与关节,并在开启状态检测嵌套物体。我们引入新数据集Arti4D-Semantic,融合62个野外场景的层级语义(含父子关系标签)与物体轴线标注,涵盖600次交互和三种观测范式。在两个数据集上全面评估,消融实验验证关键设计。真实世界实验在四足机器人和移动机械臂上表明,该语义-运动场景图能实现家庭环境中对可动物体的鲁棒操作。代码与数据见:https://momasg.cs.uni-freiburg.de。

原文摘要 · Abstract (English)

Semantics has enabled 3D scene understanding and affordance-driven object interaction. However, robots operating in real-world environments face a critical limitation: they cannot anticipate how objects move. Long-horizon mobile manipulation requires closing the gap between semantics, geometry, and kinematics. In this work, we present MoMa-SG, a novel framework for building semantic-kinematic 3D scene graphs of articulated scenes containing a myriad of interactable objects. Given RGB-D sequences containing multiple object articulations, we temporally segment object interactions and infer object motion using occlusion-robust point tracking. We then lift point trajectories into 3D and estimate articulation models using a novel unified twist estimation formulation that robustly estimates revolute and prismatic joint parameters in a single optimization pass. Next, we associate objects with estimated articulations and detect contained objects by reasoning over parent-child relations at identified opening states. We also introduce the novel Arti4D-Semantic dataset, which uniquely combines hierarchical object semantics including parent-child relation labels with object axis annotations across 62 in-the-wild RGB-D sequences containing 600 object interactions and three distinct observation paradigms. We extensively evaluate the performance of MoMa-SG on two datasets and ablate key design choices of our approach. In addition, real-world experiments on both a quadruped and a mobile manipulator demonstrate that our semantic-kinematic scene graphs enable robust manipulation of articulated objects in everyday home environments. We provide code and data at: https://momasg.cs.uni-freiburg.de.

3D场景图机器人操作运动建模多模态感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。