智能眼镜实现本地化实时障碍识别,兼顾性能与隐私。
Fully-sensorized smart-eyewear platform for on-device Machine Learning
- 采用自研架构在芯片端运行优化版YOLOv11模型
- 2.483MB内存下实现mAP50-95为24,帧率10FPS
- 多模态传感器融合,续航超113分钟,适合视障辅助
本文提出ARGO智能眼镜平台,旨在兼顾佩戴舒适性、高算力与低功耗。不同于依赖云端的方案,ARGO基于STM32N6微控制器及其集成神经网络处理单元(NPU),实现本地化机器学习,降低延迟并保护用户隐私。核心贡献在于软硬件与AI的协同设计,聚焦于部署优化后的YOLOv11模型,用于城市环境障碍物实时识别。为适配目标NPU,提出头域并行注意力(HPA)结构,在保持原始计算逻辑的同时提升加速器执行效率。模型在Walking On The Road(WOTR)数据集上训练,最终部署配置在严格内存限制下达到mAP50-95为24,内存占用仅2.483MB。平台集成RGB相机、飞行时间传感器、麦克风及环境传感器,支持10 FPS连续运行,单次续航约113分钟(200 mAh电池)。结果表明该平台具备高性能、隐私保护与社会接受度,凸显现代边缘AI需高度集成的跨学科协同设计趋势。
原文摘要 · Abstract (English)
This paper presents ARGO, a smart eyewear platform designed to bridge ergonomic comfort, high computational throughput, and energy efficiency. Unlike cloud-dependent solutions, ARGO leverages the STM32N6 microcontroller and its integrated Neural Processing Unit (NPU) to enable on-device machine learning, minimizing latency and preserving user privacy through local data processing. The primary contribution lies in the holistic co-design of hardware, firmware, and artificial intelligence, centered on the deployment of an optimized YOLOv11 model for real-time urban obstacle recognition. To ensure compatibility with the target NPU, we introduce Head-wise Parallel Attention (HPA), an architectural refinement that enables efficient accelerator execution while preserving the original computational logic. The model is trained on the Walking On The Road (WOTR) dataset, and the final deployed configuration achieves an mAP50-95 of 24 under strict memory constraints, with a memory footprint of only 2.483 MB. The platform integrates a multimodal sensor suite, RGB cameras, Time-of-Flight sensors, microphones, and ambient sensors, and delivers 10 FPS at a continuous autonomy of ~113 minutes on a 200 mAh battery. These results demonstrate the feasibility of a high-performance, privacy-preserving, and socially acceptable assistive device, and highlight how competitive edge AI solutions increasingly demand tightly integrated, multidisciplinary co-design approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。