arXiv:2411.06792cs.NEcs.AI2024-11

用基因演化策略优化脉冲神经网络,显著降参增效。

Evolving Efficient Genetic Encoding for Deep Spiking Neural Networks

  • 用共享基因编码间接优化神经元,替代传统权重
  • 参数压缩50%~80%,在多个数据集上性能提升0.21%~4.38%
  • 适合追求低功耗、高可扩展性的脉冲神经网络研究者

脉冲神经网络(SNN)通过离散信号处理和模拟脑神经元通信,提供比人工神经网络(ANN)更低能耗的替代方案。然而,现有SNN模型仍因大量时间步及网络深度与规模导致高计算开销。人类大脑约10亿神经元和1万亿突触仅由约2万个基因发育而成,启发我们设计一种高效基因编码策略,动态演化以低成本调控大规模深层SNN。为此,我们提出一种基因尺度化的SNN编码方案,引入全局共享基因交互,间接优化神经元编码而非权重,显著减少参数与能耗。进一步设计时空演化框架,优化初始连接规则。在适应度函数中引入两个动态正则化算子,分别使神经元编码分布趋于合适,提升基因交互的信息质量,大幅加速演化过程并提高效率。实验表明,该方法在相同架构下将参数压缩50%至80%,在CIFAR-10、CIFAR-100和ImageNet上性能提升0.21%至4.38%。结果表明,所提基因编码的时空演化策略在不同数据集与架构下具一致优势,显著提升效率、可扩展性与鲁棒性,验证了脑启发演化基因编码在SNN优化中的优越性。

原文摘要 · Abstract (English)

By exploiting discrete signal processing and simulating brain neuron communication, Spiking Neural Networks (SNNs) offer a low-energy alternative to Artificial Neural Networks (ANNs). However, existing SNN models, still face high computational costs due to the numerous time steps as well as network depth and scale. The tens of billions of neurons and trillions of synapses in the human brain are developed from only 20,000 genes, which inspires us to design an efficient genetic encoding strategy that dynamic evolves to regulate large-scale deep SNNs at low cost. Therefore, we first propose a genetically scaled SNN encoding scheme that incorporates globally shared genetic interactions to indirectly optimize neuronal encoding instead of weight, which obviously brings about reductions in parameters and energy consumption. Then, a spatio-temporal evolutionary framework is designed to optimize the inherently initial wiring rules. Two dynamic regularization operators in the fitness function evolve the neuronal encoding to a suitable distribution and enhance information quality of the genetic interaction respectively, substantially accelerating evolutionary speed and improving efficiency. Experiments show that our approach compresses parameters by approximately 50\% to 80\%, while outperforming models on the same architectures by 0.21\% to 4.38\% on CIFAR-10, CIFAR-100 and ImageNet. In summary, the consistent trends of the proposed genetically encoded spatio-temporal evolution across different datasets and architectures highlight its significant enhancements in terms of efficiency, broad scalability and robustness, demonstrating the advantages of the brain-inspired evolutionary genetic coding for SNN optimization.

脉冲神经网络基因编码低功耗演化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。