利用硬件噪声实现持续学习,让随机波动变记忆守护者。
Intrinsic-Noise Consolidation: A Doob-Barrier-Conditioned Diffusion Turns Analog Device Noise into a Continual-Learning Resource
- 用数学上精确的障碍条件约束权重变化,把噪声转为记忆保护力。
- 噪声越大越能保记忆,但到一定点后反而下降,形成倒U型曲线。
- 在真实芯片上验证有效,比传统方法提升15.6分,适合神经形态计算研究者。
在类脑硬件上,器件固有噪声通常降低精度。本文提出将其转化为记忆巩固资源。将每个突触的更新视为受障碍条件约束的扩散过程:权重动态被限制在不越过以已固化值为中心的记忆临界屏障。该条件扩散引入额外漂移项 σ² ∂/∂w log h,其恢复力随噪声方差增大而增强,并在屏障处发散。我们明确强调创新点:(a)首次将Doob障碍条件作为突触规则使用(此前所有h变换均用于生成建模,未用于突触);(b)提出可验证预测:增加内在噪声非单调提升任务保留率,呈现倒U型曲线,这是锚定漂移方法无法实现的。该预测经预注册检验通过。在单头Split-MNIST上(8个种子),该方法在中间最优值提升保留率10.9分(配对Wilcoxon检验,p=0.004),而匹配的OU/EWC/MESU方法均为单调上升。消融实验表明,移除条件即失效,最优值与屏障位置一致,倒U型在第二任务流及噪声进入前向传播时仍成立。随后在真实BrainScaleS-2硅片上测量固有噪声(加性、试验间独立、可通过片上平均调节),并将该规则运行于芯片训练循环中:在相同平均准确率下,障碍条件方法相比对照组保留率提升15.6分,实现稳定性-可塑性转移,非净精度提升(单个种子;保留率实测,能耗建模)。因此,模拟噪声成为数字加速器需耗能模拟的巩固红利。
原文摘要 · Abstract (English)
On analog neuromorphic hardware, intrinsic device noise is normally an accuracy tax. We ask whether it can instead consolidate memories. We cast per-synapse consolidation as a Doob h-transform: condition each weight's stochastic dynamics on never crossing a memory-critical barrier around its consolidated value. The conditioned diffusion gains an extra drift sigma^2 d/dw log h, a restoring force amplified by the noise variance itself that diverges at the barrier. We are explicit about novelty: the anchored drift -s(w-mu) our rule also contains is not ours (the limit of OUA, MESU, and EWC), and we surrender it. We claim only the conjunction of (a) the Doob barrier-conditioning as a synaptic rule, to our knowledge unclaimed (every h-transform use we found is generative modeling, none synaptic), and (b) a falsifiable prediction: increasing intrinsic noise non-monotonically improves sequential-task retention, an inverted-U that anchored-drift methods cannot produce. We pre-registered this as a go/no-go gate; it passes. On single-head Split-MNIST (8 seeds) the rule lifts retention 10.9 points at an interior optimum (paired Wilcoxon p=0.004), while matched OU/EWC/MESU anchors are monotone. Ablating the conditioning removes the effect; the optimum tracks the barrier; the inverted-U survives a second task stream and the realization where noise enters the forward pass. We then measure the intrinsic noise on real BrainScaleS-2 silicon (additive, trial-to-trial independent, tunable via on-chip averaging) and run the rule on the chip with its noise in the training loop: barrier-conditioning retains a prior task 15.6 points better than the matched control at matched average accuracy, a stability-plasticity shift, not a net-accuracy win (single seed; retention measured, energy modelled). Intrinsic analog noise thus becomes a consolidation dividend a digital accelerator must spend energy to generate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。