针对脉冲神经网络设计的新初始化方法，提升训练效率与泛化能力。
IKUN: Initialization to Keep snn training and generalization great with sUrrogate-stable variaNce
- 结合代理梯度的方差稳定初始化，优化信号传播路径。
- 训练效率提升50%，达95%准确率，泛化能力达91%。
- 适合研究脉冲神经网络或想提升模型泛化性能者使用。
权重初始化显著影响神经网络的收敛性和性能。尽管Xavier和Kaiming等传统方法广泛应用，但在脉冲神经网络（SNNs）中表现不佳，因其与人工神经网络（ANNs）有不同需求。为此，我们提出 extbf{IKUN}，一种集成代理梯度函数的方差稳定初始化方法，专为SNNs设计。该方法稳定信号传播，加速收敛并增强泛化能力。实验表明， extbf{IKUN}将训练效率提升最高50%，达到95%的训练准确率和91%的泛化准确率。赫森分析显示， extbf{IKUN}训练的模型收敛至更平坦的极小值，其赫森特征值在正半轴接近零，有利于更好泛化。该方法已开源： exttt{https://github.com/MaeChd/SurrogateVarStabe}。
原文摘要 · Abstract (English)
Weight initialization significantly impacts the convergence and performance of neural networks. While traditional methods like Xavier and Kaiming initialization are widely used, they often fall short for spiking neural networks (SNNs), which have distinct requirements compared to artificial neural networks (ANNs). To address this, we introduce \textbf{IKUN}, a variance-stabilizing initialization method integrated with surrogate gradient functions, specifically designed for SNNs. \textbf{IKUN} stabilizes signal propagation, accelerates convergence, and enhances generalization. Experiments show \textbf{IKUN} improves training efficiency by up to \textbf{50\%}, achieving \textbf{95\%} training accuracy and \textbf{91\%} generalization accuracy. Hessian analysis reveals that \textbf{IKUN}-trained models converge to flatter minima, characterized by Hessian eigenvalues near zero on the positive side, promoting better generalization. The method is open-sourced for further exploration: \href{https://github.com/MaeChd/SurrogateVarStabe}{https://github.com/MaeChd/SurrogateVarStabe}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。