用八叉树扩散模型统一实现3D场景补全、扩展与生成,跨室内室外场景通用。
Octree Diffusion for Semantic Scene Generation and Completion
- 基于双八叉树图潜空间,分两阶段生成:先建粗略结构,再生成语义标签。
- 仅需单次激光扫描即可完成高质量补全,对未知数据零样本泛化。
- 适用于机器人导航,无需微调,可直接推理时补全或扩展场景。
3D语义场景的补全、扩展与生成是机器人导航与探索中密切相关的能力。现有方法常将这些问题解耦处理,且多为领域特定,需针对不同数据分布(如室内与室外)分别建模。为统一这些技术并实现跨域兼容,我们提出单一框架Octree Latent Semantic Diffusion,可在室内与室外场景中同时完成场景补全、扩展与生成。该方法直接在高效的双八叉树图潜表示上运行:一种层次化、稀疏且内存高效的占用结构。合成过程分为两阶段:(i) 结构扩散,预测二值分割信号以构建粗略占用八叉树;(ii) 潜语义扩散,通过图变分自编码器生成语义嵌入,并解码为体素级语义标签。进行语义场景补全或扩展时,模型利用推理时的潜空间填充(inpainting)或外扩(outpainting),仅依赖部分激光雷达扫描或地图作为条件,无需重训练或微调。实验表明,该模型能从单次激光雷达扫描中生成高质量结构、一致语义,并具备强鲁棒性,且对分布外的激光雷达数据实现零样本泛化。结果表明,在双八叉树图潜空间中通过生成完成补全,是真实世界机器人感知任务中比回归式流程更实用、可扩展的替代方案。
原文摘要 · Abstract (English)
The completion, extension, and generation of 3D semantic scenes are an interrelated set of capabilities that are useful for robotic navigation and exploration. Existing approaches seek to decouple these problems and solve them one-off. Additionally, these approaches are often domain-specific, requiring separate models for different data distributions, e.g.\ indoor vs.\ outdoor scenes. To unify these techniques and provide cross-domain compatibility, we develop a single framework that can perform scene completion, extension, and generation in both indoor and outdoor scenes, which we term Octree Latent Semantic Diffusion. Our approach operates directly on an efficient dual octree graph latent representation: a hierarchical, sparse, and memory-efficient occupancy structure. This technique disentangles synthesis into two stages: (i) structure diffusion, which predicts binary split signals to construct a coarse occupancy octree, and (ii) latent semantic diffusion, which generates semantic embeddings decoded by a graph VAE into voxel-level semantic labels. To perform semantic scene completion or extension, our model leverages inference-time latent inpainting, or outpainting respectively. These inference-time methods use partial LiDAR scans or maps to condition generation, without the need for retraining or finetuning. We demonstrate high-quality structure, coherent semantics, and robust completion from single LiDAR scans, as well as zero-shot generalization to out-of-distribution LiDAR data. These results indicate that completion-through-generation in a dual octree graph latent space is a practical and scalable alternative to regression-based pipelines for real-world robotic perception tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。