arXiv:2605.26061cs.LGcs.AI2026-05被引 1

用生物神经机制设计随机注意力,让模型自动生成可信的不确定性估计。

Neuronal Stochastic Attention Circuit (NSAC) for Probabilistic Representation Learning

论文配图:Neuronal Stochastic Attention Circuit (NSAC) for Probabilistic Representation Learning
图 1 · 摘自论文原文
  • 将注意力计算建模为受输入调控的随机微分方程,引入高斯分布的逻辑-正态权重。
  • 在多种连续时间任务中精度不输基线,且能同时量化数据噪声与模型不确定。
  • 结构可解释到神经元层级,适合需要可信预测的工业与自动驾驶场景。

连续时间(CT)表征学习中的可靠不确定性量化仍处于起步阶段,尤其在连续时间注意力领域。我们提出神经随机注意力电路(NSAC),一种受生物启发的新型连续时间注意力架构。该方法将注意力对数几率的计算重构为一个由输入依赖的非线性耦合门调控的奥恩斯坦-乌伦贝克随机微分方程的解,这些门基于重用秀丽隐杆线虫神经回路策略(NCPs)的布线机制。该设计在对数几率上诱导出高斯分布,并通过逻辑-正态分布将可证明的随机性传播至注意力权重,从而生成概率化输出。采用包含高斯负对数似然与认知分离正则项的双目标函数,强制在分布偏移下提高预测方差,实现对偶然性与认知性不确定性的联合量化。理论上,我们提供了:(i) 状态稳定性边界;(ii) 闭式保证;(iii) 固定系数误差近似。实证上,我们在包括:(i) 不规则连续时间函数逼近;(ii) 多变量回归;(iii) 长程预测;(iv) 工业4.0;以及 (v) 自动驾驶车道保持在内的多样化任务中实现了NSAC。结果显示,NSAC在精度上媲美多个基线,同时生成具有信息量的不确定性估计,并可在神经元层级进行可解释性分析。

原文摘要 · Abstract (English)

Reliable uncertainty quantification in continuous-time (CT) representation learning remains nascent, particularly within CT attention literature. We introduce the Neuronal Stochastic Attention Circuit (NSAC), a novel biologically-inspired CT attention architecture that reformulates attention logit computation as the solution of an Ornstein-Uhlenbeck stochastic differential equation modulated by input-dependent, nonlinear interlinked gates derived from repurposed C. elegans Neuronal Circuit Policies (NCPs) wiring mechanism. It induces a Gaussian distribution over logits that propagates principled stochasticity through a logistic-normal distribution over attention weights to yield probabilistic output. A two-term objective function combining Gaussian negative log-likelihood with an epistemic-separation regularizer enforces higher predictive variance under distributional shifts and enables joint quantification of aleatoric and epistemic uncertainty. Theoretically, we provide: (i) state stability bounds; (ii) closed-form guarantees; and (iii) frozen-coefficient error approximation. Empirically, we implement NSAC in a diverse set of learning tasks including: (i) irregular CT function approximation; (ii) multivariate regression; (iii) long-range forecasting; (iv) Industry 4.0; and (v) lane-keeping of autonomous vehicles. We observe that NSAC remains competitive against several baselines in terms of accuracy and produces informative uncertainty estimates while being interpretable at the neuronal cell level.

注意力机制不确定性连续时间生物启发

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。