用事件触发隐式扰动，让脉冲网络在内存计算芯片上高效微调。
Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers
- 通过事件触发机制生成扰动，避免重复读写操作。
- 在CIFAR-10上达到76.41%准确率，接近随机数生成器基准。
- 适合部署于内存计算芯片的脉冲神经网络微调任务。
零阶优化仅通过前向传播评估估算梯度，适用于非可导的事件驱动脉冲神经网络（SNN）微调。然而，在内存计算（IMC）加速器上部署时，受限于显式权重扰动引发的重复读-修改-写操作，以及为实现独立每权重扰动所需的庞大随机数生成器（RNG）硬件开销。为此，我们提出隐式扰动零阶优化（IPZO）架构：事件触发扰动生成单元（PGU）计算的扰动和与IMC阵列输出加权和合并，消除扰动引起的读写操作，保持权重驻留执行。利用脉冲稀疏性，PGU仅对激活权重行生成并累加扰动，降低RNG阵列所需行维度。进一步引入地址驱动异或重组方案（PGU-XOR），缓解直接重用RNG导致的空间相关性问题。实验表明：(1) PGU-XOR在Spikingformer/CIFAR-10上准确率达76.41%（软件RNG为76.53%），在SpikeGPT/WikiText-2上困惑度（PPL）为54.20（软件RNG为53.23），而PGU-Reuse准确率下降9.56个百分点，PPL增加11.8；(2) 在TSMC 16nm CMOS工艺下，PGU-XOR相比PGU-Reuse面积开销降低40.3%-46.0%，能量开销降低15.2%-48.9%，且更快收敛，等精度下总扰动能耗仅为后者的0.51倍；(3) IPZO在批量大小B=64、时间步长T=4时，扰动能耗降至传统显式扰动的0.46x-0.83x，且随BT减小优势更显著。
原文摘要 · Abstract (English)
Zeroth-order (ZO) optimization estimates gradients using only forward-pass evaluations, making it suitable for fine-tuning non-differentiable, event-driven spiking neural networks (SNNs). However, its deployment on in-memory computing (IMC) accelerators is constrained by the repeated read-modify-write (RMW) operations arising from explicit weight perturbation and the prohibitive hardware footprint of random number generators (RNGs) for statistically independent per-weight perturbations. To address these challenges, we propose an implicit-perturbation ZO (IPZO) architecture in which perturbation sums computed by an event-triggered perturbation generation unit (PGU) are combined with the weighted sums produced by the IMC array, eliminating perturbation-induced RMW operations while preserving weight-stationary execution of IMC. By exploiting spike sparsity, the PGU generates and accumulates perturbation contributions only for spike-activated weight rows, reducing the required row dimension of the RNG array. An address-driven XOR recombination scheme (PGU-XOR) is further introduced to mitigate the spatial correlations caused by direct RNG reuse (PGU-Reuse). The results show that (1) PGU-XOR matches software RNGs in accuracy on Spikingformer/CIFAR-10 (76.41% vs. 76.53%) and perplexity (PPL) on SpikeGPT/WikiText-2 (54.20 vs. 53.23), whereas PGU-Reuse degrades accuracy by 9.56 percentage points and increases PPL by 11.8; (2) implemented in a TSMC 16-nm CMOS technology, PGU-XOR incurs 40.3%-46.0% area and 15.2%-48.9% energy overhead per matrix-vector multiplication relative to PGU-Reuse, yet its faster convergence reduces the total perturbation energy to 0.51x that of PGU-Reuse at iso-accuracy; (3) IPZO reduces the perturbation energy to 0.46x-0.83x that of conventional explicit weight perturbation for a batch size of B=64 and T=4 time steps, with the advantage growing as BT decreases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。