轻量级3D感知模型，让边缘设备实时处理点云
ESAM++: Efficient Online 3D Perception on the Edge
- 用3D稀疏特征金字塔网络替代原模型的昂贵结构
- 推理速度提升3倍，模型体积减半，精度仍具竞争力
- 适合资源受限的机器人、AR/VR等边缘场景
实时在线3D场景感知对机器人、AR/VR和自动驾驶系统至关重要，尤其在计算资源有限且注重隐私的边缘计算场景下。现有先进方法如EmbodiedSAM（ESAM）虽借助分割一切模型（SAM）实现了实时、细粒度、泛化的3D实例分割，但仍依赖计算开销巨大的3D稀疏UNet进行点云特征提取，占用了大部分3D推理时间，限制了其在资源受限设备上的实用性。本文提出ESAM++，一种专为无GPU加速的边缘设备设计的轻量级、可扩展在线3D感知方案。该方法引入3D稀疏特征金字塔网络（SFPN），高效捕捉流式3D点云的多尺度几何特征，显著降低计算开销与模型规模。我们在ScanNet、ScanNet200、SceneNN和3RScan四个挑战性分割基准上评估，结果表明，相比ESAM，ESAM++在保持竞争力精度的同时，推理速度最快提升3倍，模型尺寸缩小2倍，实现了边缘设备上的实用部署。
原文摘要 · Abstract (English)
Online 3D scene perception in real time is essential for robotics, AR/VR, and autonomous systems, particularly in edge computing scenarios where computational resources are limited and privacy is crucial. Recent state-of-the-art methods like EmbodiedSAM (ESAM) demonstrate the promise of online 3D perception by leveraging the Segment Anything Model (SAM) for real-time, fine-grained, and generalized 3D instance segmentation. However, ESAM still relies on a computationally expensive 3D sparse UNet for point cloud feature extraction, which accounts for the majority of the 3D inference time, hindering its practicality on resource-constrained devices. In this paper, we propose ESAM++, a lightweight and scalable alternative for online 3D scene perception tailored to edge devices without GPU acceleration. Our method introduces a 3D Sparse Feature Pyramid Network (SFPN) that efficiently captures multi-scale geometric features from streaming 3D point clouds while significantly reducing computational overhead and model size. We evaluate our approach on four challenging segmentation benchmarks, namely ScanNet, ScanNet200, SceneNN, and 3RScan, demonstrating that our model achieves competitive accuracy with up to 3 times faster inference with a 2 times smaller model size compared to ESAM, enabling practical deployment on edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。