arXiv:2505.05512cs.CVcs.RO2025-05被引 7

提出RoboOccWorld模型,实现室内3D场景动态预测。

Occupancy World Model for Robots

  • 结合时空感受野与引导式自回归变压器建模
  • 在ScanNet数据集上超越现有方法,精度显著提升
  • 专为室内机器人场景设计,适合具身智能研究者

场景演化理解与预测对具身智能体的探索与决策至关重要。传统方法依赖潜在实例轨迹预测,而当前工作采用占用世界模型作为生成框架,描述精细的全局场景动态。然而,现有方法主要聚焦于室外结构化道路场景,忽视了室内场景中3D占用演化的预测。本文提出一种新框架,学习观测到的细粒度占用场景演化,并构建基于联合时空感受野与引导自回归变压器的占用世界模型——RoboOccWorld。提出条件因果状态注意力(CCSA),利用下一时刻相机位姿作为条件,引导自回归变压器适应室内机器人场景。为有效利用历史观测中的时空线索,设计混合时空聚合(HSTA),基于多尺度时空窗口获取联合时空感受野。此外,基于局部标注重构了OccWorld-ScanNet基准,以支持室内3D占用场景演化预测任务评估。实验表明,RoboOccWorld在该任务上优于现有先进方法。代码即将发布。

原文摘要 · Abstract (English)

Understanding and forecasting the scene evolutions deeply affect the exploration and decision of embodied agents. While traditional methods simulate scene evolutions through trajectory prediction of potential instances, current works use the occupancy world model as a generative framework for describing fine-grained overall scene dynamics. However, existing methods cluster on the outdoor structured road scenes, while ignoring the exploration of forecasting 3D occupancy scene evolutions for robots in indoor scenes. In this work, we explore a new framework for learning the scene evolutions of observed fine-grained occupancy and propose an occupancy world model based on the combined spatio-temporal receptive field and guided autoregressive transformer to forecast the scene evolutions, called RoboOccWorld. We propose the Conditional Causal State Attention (CCSA), which utilizes camera poses of next state as conditions to guide the autoregressive transformer to adapt and understand the indoor robotics scenarios. In order to effectively exploit the spatio-temporal cues from historical observations, Hybrid Spatio-Temporal Aggregation (HSTA) is proposed to obtain the combined spatio-temporal receptive field based on multi-scale spatio-temporal windows. In addition, we restructure the OccWorld-ScanNet benchmark based on local annotations to facilitate the evaluation of the indoor 3D occupancy scene evolution prediction task. Experimental results demonstrate that our RoboOccWorld outperforms state-of-the-art methods in indoor 3D occupancy scene evolution prediction task. The code will be released soon.

3D场景预测机器人感知自回归模型占用网格

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。