首个在MCU上运行的Mamba模型，内存优化达83%。
MambaLite-Micro: Memory-Optimized Mamba Inference on MCUs
- 全用C语言实现无运行时推理引擎，融合算子并优化内存布局。
- 峰值内存降低83.0%，数值误差仅1.7×10⁻⁵，精度与PyTorch一致。
- 适配ESP32S3和STM32H7，适合边缘端序列任务部署。
由于内存有限、缺乏原生算子支持及嵌入式工具链缺失，将Mamba模型部署到微控制器（MCU）仍具挑战。本文提出首个基于Mamba的神经架构在资源受限MCU上的部署方案——完全由C语言实现、无需运行时的推理引擎MambaLite-Micro。其流程包括：(1) 将训练好的PyTorch Mamba模型权重导出为轻量格式；(2) 在C中手工实现Mamba层及配套算子，结合算子融合与内存布局优化。该方法消除大尺寸中间张量，使峰值内存降低83.0%，相对数值误差仅为1.7×10⁻⁵，与PyTorch实现保持一致。在关键词识别（KWS）和人体活动识别（HAR）任务上，分类准确率完全保留，达到100%一致性。进一步验证了跨平台可移植性，在ESP32S3与STM32H7上均稳定运行，为将Mamba等先进序列模型引入真实嵌入式场景铺平道路。
原文摘要 · Abstract (English)
Deploying Mamba models on microcontrollers (MCUs) remains challenging due to limited memory, the lack of native operator support, and the absence of embedded-friendly toolchains. We present, to our knowledge, the first deployment of a Mamba-based neural architecture on a resource-constrained MCU, a fully C-based runtime-free inference engine: MambaLite-Micro. Our pipeline maps a trained PyTorch Mamba model to on-device execution by (1) exporting model weights into a lightweight format, and (2) implementing a handcrafted Mamba layer and supporting operators in C with operator fusion and memory layout optimization. MambaLite-Micro eliminates large intermediate tensors, reducing 83.0% peak memory, while maintaining an average numerical error of only 1.7x10-5 relative to the PyTorch Mamba implementation. When evaluated on keyword spotting(KWS) and human activity recognition (HAR) tasks, MambaLite-Micro achieved 100% consistency with the PyTorch baselines, fully preserving classification accuracy. We further validated portability by deploying on both ESP32S3 and STM32H7 microcontrollers, demonstrating consistent operation across heterogeneous embedded platforms and paving the way for bringing advanced sequence models like Mamba to real-world resource-constrained applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。