提出高效神经形态持续学习方法,显著降低嵌入式系统延迟与能耗。
Replay4NCL: An Efficient Memory Replay-based Methodology for Neuromorphic Continual Learning in Embedded AI Systems
- 压缩旧知识潜空间数据,小时间步重放以减少计算开销
- 在SHD数据集上保持90.43%准确率,比现有方法高4.21个百分点
- 适合资源受限的移动机器人等嵌入式场景,兼顾性能与能效
神经形态持续学习(NCL)利用脉冲神经网络(SNNs)使AI系统能够适应动态环境。当前最先进的方法基于记忆重放,但依赖长时序和编解码步骤,导致显著延迟与能耗,不适用于资源受限的嵌入式AI系统(如移动代理/机器人)。为此,我们提出Replay4NCL,一种新型高效记忆重放方法,用于嵌入式NCL。该方法先压缩旧知识的潜空间数据,在小时间步下重放训练,以降低处理延迟与能耗;为补偿因脉冲减少造成的信息损失,调整神经元阈值电位与学习率。在类增量场景下使用脉冲海德堡数字(SHD)数据集的实验表明,Replay4NCL相比现有方法在保持旧知识方面达到90.43%的Top-1准确率(原为86.22%),同时实现4.88倍延迟加速、20%潜空间内存节省和36.43%能耗降低。结果表明,Replay4NCL具有推动嵌入式AI系统持续学习能力的潜力。
原文摘要 · Abstract (English)
Neuromorphic Continual Learning (NCL) paradigm leverages Spiking Neural Networks (SNNs) to enable continual learning (CL) capabilities for AI systems to adapt to dynamically changing environments. Currently, the state-of-the-art employ a memory replay-based method to maintain the old knowledge. However, this technique relies on long timesteps and compression-decompression steps, thereby incurring significant latency and energy overheads, which are not suitable for tightly-constrained embedded AI systems (e.g., mobile agents/robotics). To address this, we propose Replay4NCL, a novel efficient memory replay-based methodology for enabling NCL in embedded AI systems. Specifically, Replay4NCL compresses the latent data (old knowledge), then replays them during the NCL training phase with small timesteps, to minimize the processing latency and energy consumption. To compensate the information loss from reduced spikes, we adjust the neuron threshold potential and learning rate settings. Experimental results on the class-incremental scenario with the Spiking Heidelberg Digits (SHD) dataset show that Replay4NCL can preserve old knowledge with Top-1 accuracy of 90.43% compared to 86.22% from the state-of-the-art, while effectively learning new tasks, achieving 4.88x latency speed-up, 20% latent memory saving, and 36.43% energy saving. These results highlight the potential of our Replay4NCL methodology to further advances NCL capabilities for embedded AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。