让手机端3D物体检测快24倍,用多类低功耗芯片协同加速。
PointSplit: Towards On-device 3D Object Detection with Heterogeneous Low-power Accelerators
- 将点云按语义分组采样,智能分配给不同处理器
- 在手机GPU与NPU上并行处理,速度提升24.7倍
- 适合部署在带GPU和NPU的移动设备,如智能手机
在资源受限的边缘设备上运行深度学习模型备受关注,因其具备快速响应、隐私保护及无需联网仍可稳定运行的优势。当前配备多种低功耗加速器(如移动GPU与NPU)的新型边缘设备带来新机遇:过去仅靠单一加速器无法承担的重型任务,在异构加速架构下变得可行。为实现3D物体检测在该场景下的潜力,我们识别出若干技术挑战,并提出PointSplit框架。该框架包含三项核心技术:(1) 2D语义感知的偏置点采样,(2) 并行化的3D特征提取,(3) 基于角色的分组量化。我们在TensorFlow Lite上实现PointSplit,测试平台为集成移动GPU与EdgeTPU的定制硬件。在SUN RGB-D与Scannet V2两个主流RGB-D数据集上的实验表明,相较于仅使用GPU的全精度2D-3D融合3D检测器,PointSplit在多加速器设备上实现24.7倍提速,且精度相当。
原文摘要 · Abstract (English)
Running deep learning models on resource-constrained edge devices has drawn significant attention due to its fast response, privacy preservation, and robust operation regardless of Internet connectivity. While these devices already cope with various intelligent tasks, the latest edge devices that are equipped with multiple types of low-power accelerators (i.e., both mobile GPU and NPU) can bring another opportunity; a task that used to be too heavy for an edge device in the single-accelerator world might become viable in the upcoming heterogeneous-accelerator world.To realize the potential in the context of 3D object detection, we identify several technical challenges and propose PointSplit, a novel 3D object detection framework for multi-accelerator edge devices that addresses the problems. Specifically, our PointSplit design includes (1) 2D semantics-aware biased point sampling, (2) parallelized 3D feature extraction, and (3) role-based group-wise quantization. We implement PointSplit on TensorFlow Lite and evaluate it on a customized hardware platform comprising both mobile GPU and EdgeTPU. Experimental results on representative RGB-D datasets, SUN RGB-D and Scannet V2, demonstrate that PointSplit on a multi-accelerator device is 24.7 times faster with similar accuracy compared to the full-precision, 2D-3D fusion-based 3D detector on a GPU-only device.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。