超低功耗智能眼镜实现设备端实时手势识别
OpenGlass: Ultra-Low-Power On-Device AI Eyewear with Event-based Vision
- 用事件视觉+动态唤醒机制降低功耗
- 200mAh电池支持11.5小时持续运行
- 开源硬件与模型，适合嵌入式研究者
智能眼镜通过多模态传感器和本地智能实现无感上下文交互，但受限于小型化带来的功耗、内存与算力瓶颈。现有支持事件视觉与嵌入式机器学习的开源平台极为稀少。本文提出一款开源智能眼镜原型，支持快速开发新型传感器与算法。其模块化设计通过柔性FPC转接板，无需重新设计PCB即可兼容事件相机与帧相机。软硬件协同的电源管理系统结合可配置PMIC与nRF5340协调器的事件触发唤醒，使GAP9 RISC-V SoC在推理间隙保持休眠。原型实现200 mAh电池下长达11.5小时的连续设备端机器学习。以LynX数据集上极性分离事件直方图为输入，采用R(2+1)D网络在留两主体外验证中达到83.94%跨主体准确率（宏平均F1=0.781），端到端延迟78.3毫秒。时间增强与模糊类别剔除带来最大提升（+8.9个百分点）。所有硬件设计、固件与模型均开源。
原文摘要 · Abstract (English)
Smart eyewear enables unobtrusive, context-aware interaction through multimodal sensors and on-device intelligence, but is severely limited by power, memory, and compute constraints in a compact form factor. Open-hardware platforms supporting event-based vision and embedded ML at this scale are rare. This work introduces an open-source smart glasses platform for rapid prototyping of novel sensors and algorithms. Its modular design uses a flexible FPC interposer to support both event-based and frame-based cameras without full PCB redesign. A hardware-software co-designed power management system combines a configurable PMIC with event-driven wake-up via an nRF5340 coordinator, keeping the GAP9 RISC-V SoC powered down between inferences. The prototype achieves up to 11.5 hours of continuous on-device ML from a 200 mAh battery. As a demonstration, an egocentric hand gesture recognition pipeline was evaluated on the LynX dataset using polarity-separated event histograms from a Prophesee GENX320 camera. R(2+1)D achieved the best cross-subject accuracy of 83.94\% (macro F1 = 0.781) under leave-two-subjects-out validation, with 78.3 ms end-to-end inference latency on the GAP9. Temporal augmentation and removal of ambiguous classes provided the largest gains (+8.9 pp). All hardware designs, firmware, and models are released open source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。