用类脑计算提升持续学习能效,解决模型遗忘与资源受限难题
Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications

- 基于脉冲神经网络实现低功耗持续学习,适应动态变化环境
- 综合分析现有方法在能耗、内存和网络复杂度上的表现差距
- 适合关注类脑智能、边缘计算与高效学习的科研人员
深度神经网络驱动的持续学习(CL)面临计算与内存密集的挑战,亟需向更高效范式转型。类脑持续学习(NCL)通过脉冲神经网络(SNN)原理,在资源受限系统中实现高效持续学习,应对动态环境下的灾难性遗忘问题。本文系统梳理了持续学习的核心需求、设置、评估指标与场景分类,分析主流DNN方法在计算开销、内存占用与网络复杂度方面的不足,凸显能效的重要性。随后介绍低功耗类脑系统的编码方式、神经元动力学、网络结构、学习规则及软硬件平台。全面综述当前NCL前沿成果,涵盖混合监督-无监督学习、减少SNN操作、权重量化与知识蒸馏等优化技术。探讨真实应用场景进展,并展望未来开放性挑战,旨在为类脑人工智能研究社区提供参考,推动生物可解释的在线持续学习发展。
原文摘要 · Abstract (English)
The challenging deployment of compute- and memory-intensive methods from Deep Neural Network (DNN)-based Continual Learning (CL) underscores the critical need for a paradigm shift towards more efficient approaches. Neuromorphic Continual Learning (NCL) appears as an emerging solution, by leveraging the principles of Spiking Neural Networks (SNNs) which enable efficient CL algorithms executed in dynamically-changed environments with resource-constrained computing systems. Motivated by the need for a holistic study of NCL, in this survey, we first provide a detailed background on CL, encompassing the desiderata, settings, metrics, scenario taxonomy, Online Continual Learning (OCL) paradigm, recent DNN-based methods to address catastrophic forgetting (CF). Then, we analyze these methods considering CL desiderata, computational and memory costs, as well as network complexity, hence emphasizing the need for energy-efficient CL. Afterward, we provide background of low-power neuromorphic systems including encoding techniques, neuronal dynamics, network architectures, learning rules, hardware processors, software and hardware frameworks, datasets, benchmarks, and evaluation metrics. Then, this survey comprehensively reviews and analyzes state-of-the-art in NCL. The key ideas, implementation frameworks, and performance assessments are also provided. This survey covers several hybrid approaches that combine supervised and unsupervised learning paradigms. It also covers optimization techniques including SNN operations reduction, weight quantization, and knowledge distillation. Then, this survey discusses the progress of real-world NCL applications. Finally, this paper provides a future perspective on the open research challenges for NCL, since the purpose of this study is to be useful for the wider neuromorphic AI research community and to inspire future research in bio-plausible OCL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。