轻量级3D手部姿态估计框架，兼顾速度与精度，适合边缘设备部署。
SPLite Hand: Sparsity-Aware Lightweight 3D Hand Pose Estimation
- 用稀疏卷积挖掘手部图像内在稀疏性，提升效率42%。
- 新设计的SPLite解码器在树莓派5上帧率提升3.1倍。
- 支持量化训练，内存减少且精度损失极小，适合嵌入式场景。
随着AR/VR设备普及，深度学习模型在边缘设备上的部署面临实时推理、低功耗和低延迟的挑战。本文设计了一种轻量级编码器-解码器架构，通过在ResNet-18主干网络中引入稀疏卷积，利用手部姿态图像的固有稀疏性，实现端到端效率提升42%。提出SPLite解码器，在树莓派5上将解码帧率提升3.1倍，同时保持精度相当。进一步采用量化感知训练，降低内存占用，FreiHAND数据集上平均精度误差（PA-MPJPE）仅从9.0 mm增至9.1 mm。整体系统在树莓派5 CPU（BCM2712四核Arm A76）上实现2.98倍加速。在复合基准数据集上，性能媲美当前最优方法，显著提升计算效率。
原文摘要 · Abstract (English)
With the increasing ubiquity of AR/VR devices, the deployment of deep learning models on edge devices has become a critical challenge. These devices require real-time inference, low power consumption, and minimal latency. Many framework designers face the conundrum of balancing efficiency and performance. We design a light framework that adopts an encoder-decoder architecture and introduces several key contributions aimed at improving both efficiency and accuracy. We apply sparse convolution on a ResNet-18 backbone to exploit the inherent sparsity in hand pose images, achieving a 42% end-to-end efficiency improvement. Moreover, we propose our SPLite decoder. This new architecture significantly boosts the decoding process's frame rate by 3.1x on the Raspberry Pi 5, while maintaining accuracy on par. To further optimize performance, we apply quantization-aware training, reducing memory usage while preserving accuracy (PA-MPJPE increases only marginally from 9.0 mm to 9.1 mm on FreiHAND). Overall, our system achieves a 2.98x speed-up on a Raspberry Pi 5 CPU (BCM2712 quad-core Arm A76 processor). Our method is also evaluated on compound benchmark datasets, demonstrating comparable accuracy to state-of-the-art approaches while significantly enhancing computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。