在浏览器里完成微控制器视觉模型训练与部署，10分钟内搞定从采集到上线的全流程。
WebSerial Vision Training for Microcontrollers: A Browser-Based Companion to On-Device CNN Training
- 浏览器内直接刷写固件、采集图像、训练CNN模型，全程无需安装软件。
- 30张图片每类、20轮训练，浏览器端1分钟完成，比设备端快3倍。
- 适合教育、小企业及研究者在真实部署环境下快速验证视觉分类任务。
本文提出webmcu-vision-web，一个单文件、免安装的浏览器应用，用于在售价15-40美元的Seeed Studio XIAO ESP32-S3 Sense（XIAO ML Kit）上实现端到端的TinyML视觉模型训练与部署。作为论文[1]中Arduino固件的浏览器伴侣，该系统提供完全本地化的机器学习流程：从固件烧录、图像采集、CNN训练、权重导出到实时激活可视化，全程无需额外软件安装，仅需基于Chromium的浏览器。系统支持通过esptool-js在浏览器中刷写固件；内置SD卡文件浏览器，可预览并编辑图像；config.json实时同步，实现无需重新编译的超参数调整；支持摄像头与ESP32 OV2640相机图像采集；在浏览器侧完成三分类任务（每类约30张图像，20轮训练）仅需约1分钟，相比设备端节省8分钟；支持导出myWeights.bin和myWeights.h格式权重；生成混淆矩阵；并通过串流展示推理时的Conv2激活热力图。所有数据均不离开本地设备。针对三分类基准问题（0空白、1杯子、2笔）进行五次运行评估，结果表明收敛稳定，报告了平均准确率与标准差。所有成果均已开源，仓库链接见下。该仓库为LLM辅助适配新硬件与新任务的活模板。全部源码采用MIT许可，发布于https://github.com/webmcu-ai/webmcu-vision-web。
原文摘要 · Abstract (English)
This paper presents webmcu-vision-web, a single-file, zero-install browser application for end-to-end TinyML vision model training and deployment on the Seeed Studio XIAO ESP32-S3 Sense (XIAO ML Kit, $15--40 USD). Acting as a browser-based companion to the on-device Arduino firmware of Paper 1 [1], it provides a private, fully local machine learning pipeline, from firmware flashing through image collection, CNN training, weight export, and live activation visualization, without any software installation beyond a Chromium-based browser. The system targets educators, small businesses, and researchers who need to train task-specific visual classifiers under their exact deployment conditions. Key capabilities include: in-browser firmware flashing via esptool-js; an SD card file browser with image preview and inline editing; config.json live-sync for zero-recompile hyperparameter adjustment; webcam and ESP32 OV2640 camera image capture; TensorFlow.js CNN training completing a three-class run (~30 images per class, 20 epochs) in approximately 1 minute browser-side versus 9 minutes on-device, enabling a complete collect-train-deploy cycle in under 10 minutes; weight export as myWeights.bin and myWeights.h; confusion matrix; and a live Conv2 activation heatmap streamed from the ESP32 during inference. No data leaves the local machine at any stage. A five-run consistency evaluation on the three-class reference problem (0Blank, 1Cup, 2Pen) demonstrates stable convergence with mean accuracy and standard deviation reported; all artefacts are released at the repository link below. The repository is a living template for LLM-assisted adaptation to new hardware and tasks. All source code is MIT-licensed at https://github.com/webmcu-ai/webmcu-vision-web.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。