提出在线训练二值脉冲网络的新算法，大幅降低内存占用。
BSO: Binary Spiking Online Optimization Algorithm
- 通过翻转信号直接更新权重，无需存储中间权重
- 理论证明算法收敛，且在多个数据集上优于现有方法
- 适合资源受限设备的高效神经网络训练
二值脉冲神经网络（BSNNs）在资源受限计算中具有显著效率优势，但其训练算法常因隐式权重存储和时序处理需求导致高内存开销。为此，我们提出一种新型在线训练算法——二值脉冲在线优化（BSO），显著降低训练内存消耗。BSO 在在线框架下通过翻转信号直接更新权重，当梯度动量与权重乘积超过阈值时触发该信号，从而避免训练期间存储隐式权重。为提升性能，我们进一步提出时序感知变体 T-BSO，利用 BSNN 的固有时序动态，跨时间步捕捉梯度信息以自适应调整阈值。理论分析建立了 BSO 与 T-BSO 的收敛性保证，给出正式的遗憾界刻画其收敛速率。大量实验表明，两者在优化性能上均优于现有针对 BSNN 的训练方法。代码已开源：https://github.com/hamings1/BSO。
原文摘要 · Abstract (English)
Binary Spiking Neural Networks (BSNNs) offer promising efficiency advantages for resource-constrained computing. However, their training algorithms often require substantial memory overhead due to latent weights storage and temporal processing requirements. To address this issue, we propose Binary Spiking Online (BSO) optimization algorithm, a novel online training algorithm that significantly reduces training memory. BSO directly updates weights through flip signals under the online training framework. These signals are triggered when the product of gradient momentum and weights exceeds a threshold, eliminating the need for latent weights during training. To enhance performance, we propose T-BSO, a temporal-aware variant that leverages the inherent temporal dynamics of BSNNs by capturing gradient information across time steps for adaptive threshold adjustment. Theoretical analysis establishes convergence guarantees for both BSO and T-BSO, with formal regret bounds characterizing their convergence rates. Extensive experiments demonstrate that both BSO and T-BSO achieve superior optimization performance compared to existing training methods for BSNNs. The codes are available at https://github.com/hamings1/BSO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。