让3D点云采集与语义分割同步进行,大幅提速并提前出结果。
RESSCAL3D++: Joint Acquisition and Semantic Segmentation of 3D Point Clouds
- 利用可变分辨率传感器,边采集边处理点云数据。
- 推理速度提升15.6%至63.9%,精度损失仅0.2%(mIoU)。
- 首次在7%时间内完成预测,适合实时场景应用。
3D场景理解对实现数字设备与物理世界的无缝交互至关重要。实时捕获与处理3D场景是达成这一目标的关键。现有方法通常将每一帧的采集与处理分开进行,而可变分辨率3D传感器的出现提供了突破这一范式的机会,可充分利用原本浪费的采集时间启动处理。本文提出VX-S3DIS,一个精确模拟可变分辨率3D传感器行为的新点云数据集。同时,我们改进了先前工作RESSCAL3D,引入更新模块与处理策略,形成RESSCAL3D++。在新数据集上应用该方法,实际验证了点云采集与语义分割联合处理的潜力。所提方法将可扩展性成本从2%降至0.2%(mIoU),相比非可扩展基线,推理速度提升15.6%至63.9%。此外,该方法支持早期预测,首个预测仅需基线总推理时间的7%。VX-S3DIS数据集已开源:https://github.com/remcoroyen/vx-s3dis。
原文摘要 · Abstract (English)
3D scene understanding is crucial for facilitating seamless interaction between digital devices and the physical world. Real-time capturing and processing of the 3D scene are essential for achieving this seamless integration. While existing approaches typically separate acquisition and processing for each frame, the advent of resolution-scalable 3D sensors offers an opportunity to overcome this paradigm and fully leverage the otherwise wasted acquisition time to initiate processing. In this study, we introduce VX-S3DIS, a novel point cloud dataset accurately simulating the behavior of a resolution-scalable 3D sensor. Additionally, we present RESSCAL3D++, an important improvement over our prior work, RESSCAL3D, by incorporating an update module and processing strategy. By applying our method to the new dataset, we practically demonstrate the potential of joint acquisition and semantic segmentation of 3D point clouds. Our resolution-scalable approach significantly reduces scalability costs from 2% to just 0.2% in mIoU while achieving impressive speed-ups of 15.6 to 63.9% compared to the non-scalable baseline. Furthermore, our scalable approach enables early predictions, with the first one occurring after only 7% of the total inference time of the baseline. The new VX-S3DIS dataset is available at https://github.com/remcoroyen/vx-s3dis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。