轻量级状态空间模型实现低功耗设备高效人体动作识别
BabyMamba-HAR: Lightweight Selective State Space Models for Efficient Human Activity Recognition on Resource Constrained Devices
- 设计双轻量架构，支持噪声鲁棒与通道无关复杂度
- 8个数据集平均F1达86.52%，参数仅27K、计算量2.21M MACs
- 适配微控制器部署，内存占用降低至原理论的1/11
在资源受限设备上进行人体动作识别（HAR）需兼顾高精度与多样传感器配置。选择性状态空间模型（SSMs）提供线性时间序列处理，是注意力机制的高效替代。然而其在TinyML领域的设计空间尚未探索。本文提出BabyMamba-HAR，包含两种轻量架构：(1) CI-BabyMamba-HAR，采用通道独立茎干提升抗噪能力；(2) Crossover-BiDir-BabyMamba-HAR，通过早期融合茎干实现通道数无关的复杂度。两者均集成权重共享双向扫描与门控时序注意力池化。在八个基准测试中，Crossover-BiDir-BabyMamba-HAR平均F1得分86.52%，参数量27K，计算量2.21M MACs，性能媲美TinyHAR（86.16%），但高通道数据集下计算量减少11倍。在Raspberry Pi Pico 2和ESP32上使用混合精度C++运行时（INT8投影，float32状态）部署，融合计算策略结合生命周期感知内存管理，将峰值内存从O(B×d_model×L×d_state)降至O(B×d_model×d_state)，支持权重共享双向与通道流执行。两种架构均实现8/8数据集覆盖，且超过99.2%与PyTorch兼容性；而量化后的TFLite基线表现下降（如TinyHAR：7/8覆盖，4/8兼容性），DeepConvLSTM在两平台上分别仅达1/8和0/8覆盖。在ESP32上平均延迟154.4毫秒，在Pico 2上为481.9毫秒。消融实验表明双向扫描与门控注意力可分别提升F1最高达8.42%与8.94%，确立了适用于TinyML SSM部署的实用原则。
原文摘要 · Abstract (English)
Human activity recognition (HAR) on resource constrained devices requires high accuracy across diverse sensor setups. Selective state space models (SSMs) offer efficient linear time sequence processing, presenting a compelling alternative to attention mechanisms. However, their TinyML design space remains unexplored. This paper introduces BabyMamba-HAR, comprising two lightweight architectures: (1) CI-BabyMamba-HAR, utilizing a channel independent stem for noise robustness, and (2) Crossover-BiDir-BabyMamba-HAR, utilizing an early fusion stem for channel count independent complexity. Both integrate weight tied bidirectional scanning and gated temporal attention pooling. Across eight benchmarks, Crossover-BiDir-BabyMamba-HAR averages an 86.52% F1-score with 27K parameters and 2.21M MACs, matching TinyHAR (86.16%) while requiring 11x fewer MACs on high channel datasets. On-device deployment on the Raspberry Pi Pico 2 and ESP32 utilized a mixed precision C++ runtime (INT8 projections, float32 states). A fused computation strategy with lifetime aware memory management reduces peak memory footprint from O(B*dmodel*L*dstate) to O(B*dmodel*dstate), adapting to support weight-tied bidirectional and channel-streaming execution. Both architectures achieved full 8/8 dataset coverage with >99.2% PyTorch parity, whereas INT8 quantized TFLite baselines showed degraded coverage and parity (TinyHAR: 7/8 and 4/8 coverage at 60.4% and 88.6% parity, TinierHAR: 8/8 and 6/8 at 54.2% and 90.8%, DeepConvLSTM: 1/8 and 0/8 on Pico 2 and ESP32, respectively). Crossover-BiDir-BabyMamba-HAR averages 154.4 ms latency on ESP32 and 481.9 ms on Pico 2. Ablations confirm bidirectional scanning and gated attention improve F1-scores by up to 8.42% and 8.94%, respectively, establishing practical principles for TinyML SSM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。