用六面投影结构高效处理3D点云,提升语义理解精度。
HexPlane Representation for 3D Semantic Scene Understanding
- 将点云投射到六个平面,用2D网络提取特征并自适应融合。
- 在ScanNet上达77.0 mIoU,比Point Transformer V2高1.6。
- 可无缝接入现有方法,无需额外调参即获显著提升。
本文提出HexPlane表示用于3D语义场景理解。首先设计视图投影模块(VPM)将3D点云投影至六个平面,最大限度保留原始空间信息;通过2D编码器提取各平面特征,并送入HexPlane关联模块(HAM)自适应融合每点最相关信息;融合后的点特征输入任务头生成最终预测。相比主流点和体素表示,HexPlane高效利用高度优化的2D操作处理稀疏无序的3D点云,可直接复用现成的2D模型、权重与训练方案,实现精准3D场景理解。在ScanNet和SemanticKITTI基准上,所提方法HexNet3D表现优异:在ScanNet 3D分割任务中验证集mIoU达77.0,超越Point Transformer V2 1.6;室内3D检测任务也取得令人鼓舞的结果。该方法可无缝集成至现有体素、点和范围基方法中,不需复杂调整即可带来明显性能提升。代码将在论文发表后公开。
原文摘要 · Abstract (English)
In this paper, we introduce the HexPlane representation for 3D semantic scene understanding. Specifically, we first design the View Projection Module (VPM) to project the 3D point cloud into six planes to maximally retain the original spatial information. Features of six planes are extracted by the 2D encoder and sent to the HexPlane Association Module (HAM) to adaptively fuse the most informative information for each point. The fused point features are further fed to the task head to yield the ultimate predictions. Compared to the popular point and voxel representation, the HexPlane representation is efficient and can utilize highly optimized 2D operations to process sparse and unordered 3D point clouds. It can also leverage off-the-shelf 2D models, network weights, and training recipes to achieve accurate scene understanding in 3D space. On ScanNet and SemanticKITTI benchmarks, our algorithm, dubbed HexNet3D, achieves competitive performance with previous algorithms. In particular, on the ScanNet 3D segmentation task, our method obtains 77.0 mIoU on the validation set, surpassing Point Transformer V2 by 1.6 mIoU. We also observe encouraging results in indoor 3D detection tasks. Note that our method can be seamlessly integrated into existing voxel-based, point-based, and range-based approaches and brings considerable gains without bells and whistles. The codes will be available upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。