在微型红外传感器上实现低功耗实时手势识别与自适应更新
MAUPITI: On-Device Prototype-Based Learning on a Smart Infrared Sensor
- 用原型法替代反向传播，节省内存与功耗
- 精度媲美传统模型，更新延迟低于0.29%
- 适合资源受限的嵌入式隐私感知设备
低分辨率红外阵列传感器为嵌入式系统中的隐私保护人体感知提供了新思路。本文介绍了一种集成16×16热MOSFET（TMOS）阵列与扩展了低精度SIMD指令的RISC-V微控制器的智能多像素红外传感器，可在极低内存（<32kB片上存储）和功耗（≈1.5mW）约束下实现设备端学习与持续适应，用于姿态与手势识别。为避免反向传播和回放缓冲区带来的内存开销，采用基于原型的最近类均值（NCM）分类器：简单卷积神经网络（CNN）编码器离线训练并量化，类别原型在设备端以流式模式存储并更新。在两个数据集上的实验表明，该方法在分类精度上与传统分类器相当，分类与原型更新阶段总延迟低于0.29%，有效实现了感知框架的在线自适应。
原文摘要 · Abstract (English)
Low-resolution infrared (IR) array sensors represent an interesting solution for privacy-preserving human sensing in embedded systems. In this letter, we describe a smart multi-pixel IR sensor integrating a 16$\times$16 thermal MOSFET (TMOS) array and a RISC-V microcontroller extended with low-precision SIMD instructions, capable of on-device learning and continual adaptation for pose and gesture recognition tasks under tight memory and power constraints ($<$32kB on-chip memory, $\approx$1.5mW). To avoid the memory overheads of backpropagation and replay buffers, we adopt a prototype-based Nearest Class Mean (NCM) classifier in which a simple Convolutional Neural Network (CNN) encoder is trained and quantized offline, while class prototypes are stored and updated on the device in streaming mode. With experiments on two datasets, we show that this approach yields accuracy on par with a conventional classifier, with negligible latency overheads in both the classification and the prototype update ($<$0.29% considering both phases), effectively enabling online adaptation of the perception framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。