arXiv:2503.07107cs.LGcs.CV2025-03

让全二值神经网络实现增量学习,突破边缘设备的持续推理瓶颈。

Towards Experience Replay for Class-Incremental Learning in Fully-Binary Networks

  • 重构全二值网络设计,适配类增量学习场景
  • 3Mb模型在CORE50上性能媲美更大实值模型
  • 提出半监督预训练与损失平衡策略,提升迁移能力

全二值神经网络(FBNN)是实现超低功耗边缘设备上神经网络部署的有前景方案。此类设备常运行于动态环境,需应对类别演变或新类出现,要求持续学习能力。类增量学习(CIL)是分类任务中常见的持续学习范式,但在FBNN中研究极少。此外,多数现有BNN仍非完全二值,因输入、输出及批量归一化层需实值计算。本文进一步推进全二值化,在此基础上实现类增量学习,主要贡献包括:重新审视适合CIL的FBNN架构与训练流程;探索损失平衡机制,权衡旧类与新类性能;提出半监督方法预训练特征提取器以获得可迁移表征;系统比较两种经典CIL方法(隐空间重放与原生重放)。实验首先在CIFAR100上验证,再扩展至CORE50基准。最终基于3Mb FBNN模型在CORE50上表现达到甚至优于传统更大规模实值模型。

原文摘要 · Abstract (English)

Binary Neural Networks (BNNs) are a promising approach to enable Artificial Neural Network (ANN) implementation on ultra-low power edge devices. Such devices may compute data in highly dynamic environments, in which the classes targeted for inference can evolve or even novel classes may arise, requiring continual learning. Class Incremental Learning (CIL) is a common type of continual learning for classification problems, that has been scarcely addressed in the context of BNNs. Furthermore, most of existing BNNs models are not fully binary, as they require several real-valued network layers, at the input, the output, and for batch normalization. This paper goes a step further, enabling class incremental learning in Fully-Binarized NNs (FBNNs) through four main contributions. We firstly revisit the FBNN design and its training procedure that is suitable to CIL. Secondly, we explore loss balancing, a method to trade-off the performance of past and current classes. Thirdly, we propose a semi-supervised method to pre-train the feature extractor of the FBNN for transferable representations. Fourthly, two conventional CIL methods, \ie, Latent and Native replay, are thoroughly compared. These contributions are exemplified first on the CIFAR100 dataset, before being scaled up to address the CORE50 continual learning benchmark. The final results based on our 3Mb FBNN on CORE50 exhibit at par and better performance than conventional real-valued larger NN models.

二值网络增量学习边缘计算持续学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。