arXiv:2603.13849cs.LG2026-03

提出可学习不确定性的神经元,让每个神经元自带概率结构。

Exploring the Dimensions of a Variational Neuron

  • 将概率建模从全局参数移到单个神经元,用隐变量控制其不确定性。
  • 实验发现神经元隐空间维度(k)影响其学习模式,高维更稳定。
  • 适合研究模型不确定性、可解释性及鲁棒训练的开发者参考。

我们提出EVE(Elemental Variational Expanse),一种基于显式先验与摊销后验的变分分布神经元,作为局部概率计算单元。传统模型通过全局潜在变量或参数不确定性建模置信度,而计算单元本身仍是标量。EVE将概率结构置于神经元层面,实现局部可观测与可控。本文中“维度”主要指神经元内部潜在维度k,从原子情况k=1到高维隐空间变化。研究揭示k值如何改变神经元的学习状态,并考察其与局部容量控制及神经元级自回归扩展带来的时序持续性的交互作用。为支持研究,EVE引入内嵌诊断工具,包括有效KL、mu²目标区间、越界比例、漂移与坍塌指标。在多个预测与表格数据任务中,结果表明潜在维度、控制机制和时序扩展共同塑造神经元内部状态,部分神经元变量可测量、具信息量且与下游性能相关。整体上,本工作首次以实验为基础,绘制了变分神经元设计空间的初步地图。

原文摘要 · Abstract (English)

We introduce EVE (Elemental Variational Expanse), a variational distributional neuron formulated as a local probabilistic computational unit with an explicit prior, an amortized posterior, and unit-level variational regularization. In most modern architectures, uncertainty is modeled through global latent variables or parameter uncertainty, while the computational unit itself remains scalar. EVE instead relocates probabilistic structure to the neuron level, making it locally observable and controllable. In this paper, the term dimensions refers primarily to the neuron's internal latent dimensionality, denoted by k. We study how varying k, from the atomic case k = 1 to higher-dimensional latent spaces, changes the neuron's learned operating regime. We then examine how this main axis interacts with two additional structural properties: local capacity control and temporal persistence through a neuron-level autoregressive extension. To support this study, EVE is instrumented with internal diagnostics and constraints, including effective KL, a target band on mu^2, out-of-band fractions, and indicators of drift and collapse. Across selected forecasting and tabular settings, we show that latent dimensionality, control, and temporal extension shape the neuron's internal regime, and that some neuron-level variables are measurable, informative, and related to downstream behavior. Overall, the paper provides an experimentally grounded first map of the design space opened by a variational neuron.

变分神经元不确定性建模可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。