通过周期性调整精度，让二值神经网络训练更快更省电。
CycleBNN: Cyclic Precision Training in Binary Neural Networks
- 用周期性切换精度的方式优化二值网络训练效率
- 在ImageNet上训练操作量减少96.09%
- 适合资源受限设备上的高效模型训练
本文研究二值神经网络（BNNs），这类网络能显著降低计算开销和内存占用。但训练能耗高、性能下降仍是难题。不同于以往仅关注推理优化的工作，本文提出一种新方法——CycleBNN，将二值网络与周期性精度训练结合，通过动态循环调整精度，在训练效率与模型性能间实现良好平衡。该方法特别适用于数据本地采集的能源受限场景，推动可持续高效的深度学习架构发展。在ImageNet、CIFAR-10和PASCAL-VOC上的实验表明，其性能具有竞争力，同时在ImageNet上训练操作量减少96.09%，CIFAR-10上减少88.88%，PASCAL-VOC上减少96.09%。该方法为高效网络的快速可及训练提供了路径。代码已开源：https://github.com/fedeloper/CycleBNN/
原文摘要 · Abstract (English)
This paper works on Binary Neural Networks (BNNs), a promising avenue for efficient deep learning, offering significant reductions in computational overhead and memory footprint to full precision networks. However, the challenge of energy-intensive training and the drop in performance have been persistent issues. Tackling the challenge, prior works focus primarily on task-related inference optimization. Unlike prior works, this study offers an innovative methodology integrating BNNs with cyclic precision training, introducing the CycleBNN. This approach is designed to enhance training efficiency while minimizing the loss in performance. By dynamically adjusting precision in cycles, we achieve a convenient trade-off between training efficiency and model performance. This emphasizes the potential of our method in energy-constrained training scenarios, where data is collected onboard and paves the way for sustainable and efficient deep learning architectures. To gather insights on CycleBNN's efficiency, we conduct experiments on ImageNet, CIFAR-10, and PASCAL-VOC, obtaining competitive performances while using 96.09\% less operations during training on ImageNet, 88.88\% on CIFAR-10 and 96.09\% on PASCAL-VOC. Finally, CycleBNN offers a path towards faster, more accessible training of efficient networks, accelerating the development of practical applications. The PyTorch code is available at \url{https://github.com/fedeloper/CycleBNN/}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。