在15-40美元的微型控制器上实现完整视觉模型训练与实时推理。
On-Device Vision Training, Deployment, and Inference on a Thumb-Sized Microcontroller
- 全流程用C++在微控制器上实现，无需外部依赖。
- 每轮训练约9分钟，推理速度达6.3帧/秒。
- 适合嵌入式开发、边缘AI初学者快速上手。
本文提出一个完整的端侧视觉机器学习流水线，涵盖数据采集、两层CNN训练（使用Adam优化）及实时推理，全部在成本15-40美元的微控制器上运行。与依赖外部基础设施的云端工作流不同，该系统以约1750行可读性C++代码实现核心机器学习生命周期，可在Arduino IDE中一分钟内编译完成，且无外部机器学习依赖。系统运行于Seeed Studio ESP32-S3 XIAO ML Kit（8 MB PSRAM），实现三分类64x64图像识别，每轮训练耗时约9分钟，推理速度达6.3 FPS。关键技术包括：批级梯度累积正确实现；推理时预计算缩放查表；支持SD-free嵌入式部署的双格式权重导出；三重权重优先级系统（SD二进制 > 嵌入头文件 > He初始化）自动加载；单常量网络重构接口；以及适配微控制器内存约束的PSRAM感知内存管理。所有源码与参考数据集已开源，许可为MIT，地址：https://github.com/webmcu-ai/on-device-vision-ai。
原文摘要 · Abstract (English)
This paper presents a complete, end-to-end on-device vision machine learning pipeline, comprising data acquisition, two-layer CNN training with Adam optimization, and real-time inference, executing entirely on a microcontroller-class device costing $15-40 USD. Unlike cloud-based workflows that require external infrastructure and conceal the computational pipeline from the practitioner, this system implements every step of the core ML lifecycle in approximately 1,750 lines of readable C++ that compiles in under one minute using the Arduino IDE, with no external ML dependencies. Running on the Seeed Studio ESP32-S3 XIAO ML Kit (8 MB PSRAM), the firmware achieves three-class 64x64 image classification in approximately 9 minutes per training run, with real-time inference at 6.3 FPS. Key contributions include: correct batch-level gradient accumulation; pre-computed resize lookup tables for inference; dual-format weight export for SD-free baked-in deployment; a three-tier weight priority system (SD binary > baked-in header > He-initialization) resolved automatically at boot; a single-constant network reconfiguration interface; and PSRAM-aware memory management suited to microcontroller constraints. All source code and reference datasets are released under the MIT License at https://github.com/webmcu-ai/on-device-vision-ai
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。