用深度学习将室内点云转为紧凑多立方体表示，提升场景重建效率。
Deep Polycuboid Fitting for Compact 3D Representation of Indoor Scenes
- 通过变换器检测六类立方体面，再用图神经网络验证空间关系
- 在真实场景数据集上实现高精度多立方体拟合，支持家具重排等应用
- 适合需要轻量化3D场景表示的虚拟现实与编辑任务
本文提出一种基于深度学习的多立方体拟合框架，用于紧凑表示3D室内场景。室内场景主要由具有直角几何特征的家具等人工物体构成，可通过多立方体组合实现高效建模，有利于后续的家具重排等应用。该框架以噪声点云为输入，首先利用变换器网络检测六类立方体面；随后采用图神经网络验证检测面之间的空间关系，生成潜在多立方体；最后基于聚合的面标签构建每个立方体实例。为训练网络，我们构建了一个涵盖多样化立方体与多立方体形状的合成数据集，能反映真实室内场景特征。该方法在Replica、ScanNet及iPhone采集的真实场景数据集上均表现出良好泛化能力。其多功能性通过虚拟房间导览和场景编辑等实际应用得以验证。
原文摘要 · Abstract (English)
This paper presents a novel framework for compactly representing a 3D indoor scene using a set of polycuboids through a deep learning-based fitting method. Indoor scenes mainly consist of man-made objects, such as furniture, which often exhibit rectilinear geometry. This property allows indoor scenes to be represented using combinations of polycuboids, providing a compact representation that benefits downstream applications like furniture rearrangement. Our framework takes a noisy point cloud as input and first detects six types of cuboid faces using a transformer network. Then, a graph neural network is used to validate the spatial relationships of the detected faces to form potential polycuboids. Finally, each polycuboid instance is reconstructed by forming a set of boxes based on the aggregated face labels. To train our networks, we introduce a synthetic dataset encompassing a diverse range of cuboid and polycuboid shapes that reflect the characteristics of indoor scenes. Our framework generalizes well to real-world indoor scene datasets, including Replica, ScanNet, and scenes captured with an iPhone. The versatility of our method is demonstrated through practical applications, such as virtual room tours and scene editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。