arXiv:2505.18069cs.LGeess.SP2025-05

L2正则化会诱导学习信号向海布式方向对齐,非学习过程也会出现类似现象。

Ubiquity of Emergent Hebbian Dynamics in Regularized Learning

  • L2正则化使更新信号趋向海布方向,强度越高越明显。
  • 即使在非学习或随机更新中,也能提前出现海布式特征。
  • 该机制可与真实海布学习共存,需新实验区分二者。

海布和反海布可塑性广泛存在于大脑中,经典模型将其视为由稳态约束稳定的局部同突触规则。这引发了一个可识别性问题:在突触更新中观察到海布/反海布结构,是否一定意味着存在底层的海布计算?我们揭示了一种替代的、涌现的路径。我们证明,在接近平稳状态时,L2权重衰减会普遍驱动多种更新规则中的学习信号成分与海布方向对齐,且对齐程度随衰减强度单调增加。这种海布式特征并非仅限于SGD,甚至在学习停止前,非学习或随机更新规则中也可出现。我们还发现,学习信号中的随机噪声可诱发反海布对齐,从而在回归设置中形成与权重衰减的简单权衡关系及相变边界。这些机制不取代标准海布理论;它们可与真实的海布可塑性共存,并使突触测量的解释复杂化,因此需要设计新实验以区分机制性海布计算与涌现的海布信号。

原文摘要 · Abstract (English)

Hebbian and anti-Hebbian plasticity are widely observed in the brain and are classically modeled as mechanistic, local homosynaptic rules stabilized by homeostatic constraints. This raises an identifiability question: does observing Hebbian/anti-Hebbian structure in synaptic updates uniquely imply an underlying Hebbian computation? We identify an alternative, emergent route. We show that near stationarity, L2 weight decay generically drives the \emph{learning-signal} component of many update rules to align with a Hebbian direction, with alignment increasing monotonically with decay strength. This Hebbian-like signature is not specific to SGD and can arise even for non-learning or random update rules long before learning has ceased. We further show that stochastic noise in the learning signal can induce anti-Hebbian alignment, yielding a simple tradeoff with weight decay and a phase boundary in regression settings. These mechanisms do not replace standard Hebbian theory; they can coexist with genuine Hebbian plasticity and complicate the interpretation of synaptic measurements, motivating experiments that distinguish mechanistic Hebbian computation from emergent Hebbian signatures.

正则化学习动力学神经科学机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。