从部分观测环境生成带层级结构的3D场景图,助力机器人探索与导航。
3D Scene Graph Prediction: Generating Hierarchical Models from Partially Observed Environments

- 分层建模:先预测房间布局与通行性,再生成室内物体分布。
- 新模型支持不完整平面图输入,无需训练时就提供残缺数据。
- 实测在未知场景下泛化能力显著优于传统方法,适合机器人应用。
生成逼真的3D室内场景是计算机视觉与机器人领域的研究热点。现有方法多聚焦单个房间的物体布局生成,而对房间级布局和通行性等高层结构生成仍缺乏关注,但后者对机器人任务至关重要。本文研究机器人部分探索环境后,预测未探查区域的问题。提出一种自顶向下的层次化3D场景图生成框架,包含描述地板布局与通行性的房间层,以及各房间内物体布局的对象层。房间层采用新型混合域图扩散模型,联合预测房间类别、地板边界及房间间通行性;通过引入破坏与掩码机制,可处理不完整平面图,避免需使用部分观测训练数据。对象层整合现有混合离散-连续扩散模型,基于已知地板图联合预测物体类别、位置、尺寸与朝向。在标准基准上对比了最先进的基于占用网格和大语言模型的平面图生成方法。结果表明,相比基于占用网格的学习基线,本方法在分布外的不完整平面图上泛化能力显著提升。还在机器人采集的真实场景数据上验证了集成预测流程,实现了对已探索区域之外的可靠预测。
原文摘要 · Abstract (English)
Generating realistic 3D indoor scenes is an area of growing interest in computer vision and robotics. Existing methods, often motivated by applications such as interior design, generally focus on object layout generation within a single room. The generation of high-level scene structure, such as room-level layout and traversability, remains underexplored despite its importance for robotics applications. In this paper, we consider the case where a robot has explored part of an environment and needs to predict the unexplored parts to support downstream tasks such as exploration or object search. We propose a top-down framework for synthesizing hierarchical 3D scene graphs, including a room layer -- describing the floor plan and traversability -- and an object layer modeling object layouts within each room. For the room layer, we propose a novel mixed-domain graph diffusion model jointly predicting room categories, floor boundaries, and traversability between rooms. Via corruption and masking, this model supports partial constraints such as incomplete floor plans, avoiding the need for partially observed training data. For the object layer, we integrate an existing mixed discrete-continuous diffusion model for joint prediction of object categories, locations, sizes, and orientations within each room given the floor plan. We compare our method with state-of-the-art occupancy-based and LLM-based floor plan generation methods on a standard benchmark. Compared with an occupancy-based learning baseline, our method generalizes substantially better to out-of-distribution partial floor plans. We also demonstrate our integrated prediction pipeline on real-world scenes from robot-collected data, enabling prediction beyond explored areas.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。