用微型传感器融合实现低功耗手势识别，适合智能眼镜等可穿戴设备。
Efficient Sensor Fusion for Gesture Recognition on Resource-Constrained Devices
- 融合低分辨率深度与热成像传感器，用轻量CNN在微控制器上实时处理。
- 92.3%准确率，0.93宏F1分数，显著优于单一传感器方案。
- 仅6343参数，50mW功耗，毫秒级延迟，适合资源受限设备。
手势识别是智能眼镜人机交互的核心技术，支持增强现实环境下的自然、无设备操控。传统视觉方法存在功耗高、计算延迟大和用户隐私泄露等问题。本文提出一种轻量级、隐私保护的手势识别系统，融合低分辨率飞行时间（ToF）与红外（IR）热成像传感器。采用8×8多区域ToF传感器（VL53L8CH）与8×8红外阵列（AMG8833）捕捉互补的深度与热信号。设计了一种具有专用分组卷积结构的紧凑型卷积神经网络（CNN），高效融合多模态信息于微控制器（MCU）上。在自建的7类静态手势数据集上，经k折交叉验证，融合策略显著优于单传感器基线，达到92.3%准确率与0.93宏F1分数。在STM32F4与STM32H7 MCU上的本地基准测试表明，该系统适用于资源受限的可穿戴设备，仅需6,343个参数，推理延迟达毫秒级，整体系统功耗为50mW。
原文摘要 · Abstract (English)
Gesture recognition is a cornerstone of Human-Computer Interaction (HCI) for smart eyewear, enabling natural and device-free control in augmented reality environments. Traditional vision-based approaches face significant challenges regarding power consumption, computational latency, and user privacy. This paper proposes a lightweight, privacy-preserving gesture recognition system based on the fusion of low-resolution Time-of-Flight (ToF) and Infrared (IR) thermal sensors. We used an 8 times 8 multizone ToF sensor (VL53L8CH) and an 8 times 8 IR array (AMG8833) to capture complementary depth and thermal cues. A compact Convolutional Neural Network (CNN) with a specialized grouped-convolution architecture is designed to fuse these modalities efficiently on a microcontroller (MCU). Experimental results on a custom dataset of 7 static gestures, validated via k-fold cross-validation, demonstrate that the proposed fusion strategy significantly outperforms single-sensor baselines with an accuracy of 92.3% and a macro F1-score of 0.93. Finally, on-device benchmarks on STM32F4 and STM32H7 MCUs confirm the system's suitability for resource-constrained wearables, requiring only 6,343 parameters and achieving millisecond-level inference latency with a total system power of 50 mW.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。