TinyPropv2让低功耗设备也能高效训练神经网络，省电又保准。
Advancing On-Device Neural Network Training with TinyPropv2: Dynamic, Sparse, and Efficient Backpropagation
- 动态调整稀疏度，可跳过部分训练步骤
- 计算量最低仅需全量训练的10%，准确率损失<1%
- 适合资源受限的物联网设备实时学习
本研究提出TinyPropv2，一种专为低功耗微控制器单元设计的设备端深度神经网络训练算法。通过动态调节反向传播中的稀疏程度，包括选择性跳过训练步骤，显著降低计算开销而不明显影响精度。在CIFAR10、CIFAR100、Flower、Food、Speech Command、MNIST、HAR及DCASE2020等多个数据集上的评估显示，其平均准确率下降仅约1个百分点。例如，在CIFAR10上仅下降0.82%，在CIFAR100上下降1.07%。在计算效率方面，某些场景下所需计算量仅为全量训练的10%，且始终优于其他稀疏训练方法。结果表明，TinyPropv2可在保持高精度的同时高效管理资源，是物联网生态中嵌入式应用的理想解决方案。
原文摘要 · Abstract (English)
This study introduces TinyPropv2, an innovative algorithm optimized for on-device learning in deep neural networks, specifically designed for low-power microcontroller units. TinyPropv2 refines sparse backpropagation by dynamically adjusting the level of sparsity, including the ability to selectively skip training steps. This feature significantly lowers computational effort without substantially compromising accuracy. Our comprehensive evaluation across diverse datasets CIFAR 10, CIFAR100, Flower, Food, Speech Command, MNIST, HAR, and DCASE2020 reveals that TinyPropv2 achieves near-parity with full training methods, with an average accuracy drop of only around 1 percent in most cases. For instance, against full training, TinyPropv2's accuracy drop is minimal, for example, only 0.82 percent on CIFAR 10 and 1.07 percent on CIFAR100. In terms of computational effort, TinyPropv2 shows a marked reduction, requiring as little as 10 percent of the computational effort needed for full training in some scenarios, and consistently outperforms other sparse training methodologies. These findings underscore TinyPropv2's capacity to efficiently manage computational resources while maintaining high accuracy, positioning it as an advantageous solution for advanced embedded device applications in the IoT ecosystem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。