arXiv:2602.11206cs.LGcs.AI2026-02

用数学新方法让脉冲神经网络可训练,省电又高效。

UltraLIF: Fully Differentiable Spiking Neural Networks via Ultradiscretization and Max-Plus Algebra

  • 用超离散化和最大-加代数替代传统近似梯度,实现完全可微。
  • 在6个基准上优于传统方法,尤其在单步输入任务中提升显著。
  • 适合需要低功耗、高效率的神经形态计算场景研究者。

脉冲神经网络(SNN)具有节能且生物合理计算的优势,但其脉冲生成过程不可微,通常依赖启发式近似梯度。本文提出UltraLIF,一种基于热带几何中超离散化的原理性框架,提供离散动态的连续松弛。核心思想是:超离散化所依赖的最大-加半环天然建模神经元阈值行为,log-sum-exp函数作为可微的软最大值,在可学习温度参数ε→0时收敛至硬阈值。从两类动力系统推导出两种神经元模型:基于积分-发放微分方程的UltraLIF(时间动态)和基于间隙连接扩散方程的UltraDLIF(空间动态)。二者均实现完全可微,支持标准反向传播且无前向-反向不匹配。理论分析证明其点态收敛至经典LIF动态,并给出量化误差界与非消失梯度边界。在六个基准测试(涵盖静态图像、神经形态视觉与音频)上的实验表明,相较传统近似梯度基线有提升,尤其在神经形态与时间数据集的单步输入(T=1)设置下表现更优。可选稀疏性惩罚可大幅降低能耗,同时保持竞争力准确率。

原文摘要 · Abstract (English)

Spiking Neural Networks (SNNs) offer energy-efficient, biologically plausible computation but suffer from non-differentiable spike generation, necessitating reliance on heuristic surrogate gradients. This paper introduces UltraLIF, a principled framework that replaces surrogate gradients with ultradiscretization, a mathematical formalism from tropical geometry providing continuous relaxations of discrete dynamics. The central insight is that the max-plus semiring underlying ultradiscretization naturally models neural threshold dynamics: the log-sum-exp function serves as a differentiable soft-maximum that converges to hard thresholding as a learnable temperature parameter $\eps \to 0$. Two neuron models are derived from distinct dynamical systems: UltraLIF from the LIF ordinary differential equation (temporal dynamics) and UltraDLIF from the diffusion equation modeling gap junction coupling across neuronal populations (spatial dynamics). Both yield fully differentiable SNNs trainable via standard backpropagation with no forward-backward mismatch. Theoretical analysis establishes pointwise convergence to classical LIF dynamics with quantitative error bounds and bounded non-vanishing gradients. Experiments on six benchmarks spanning static images, neuromorphic vision, and audio demonstrate improvements over surrogate gradient baselines, with gains most pronounced in single-timestep ($T{=}1$) settings on neuromorphic and temporal datasets. An optional sparsity penalty enables significant energy reduction while maintaining competitive accuracy.

脉冲神经网络可微计算神经形态计算最大-加代数

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。