用模拟器件噪声训练贝叶斯神经网络，结果与分布形状无关
Analog Bayesian neural networks are insensitive to the shape of the weight distribution
- 直接使用真实模拟器件的噪声作为变分分布
- 相同均值方差下预测分布趋同，与噪声形状无关
- 简化硬件设计，适合模拟芯片实现贝叶斯模型
近期研究证明，通过均场变分推断（MFVI）训练的贝叶斯神经网络（BNN）可在模拟硬件上实现，相比传统数字实现可节省数量级能耗。然而，尽管通常使用高斯分布作为变分分布，但模拟器件采样产生的噪声分布形状难以精确控制。本文提出一种直接使用真实设备噪声作为变分分布的MFVI训练方法。我们实证表明，具有相同权重均值和方差的BNN，其预测分布会收敛至相同结果，无论变分分布的形状如何。该结果表明，在实现执行MFVI的模拟贝叶斯神经网络时，硬件设计者无需关注器件噪声分布的形状。
原文摘要 · Abstract (English)
Recent work has demonstrated that Bayesian neural networks (BNN's) trained with mean field variational inference (MFVI) can be implemented in analog hardware, promising orders of magnitude energy savings compared to the standard digital implementations. However, while Gaussians are typically used as the variational distribution in MFVI, it is difficult to precisely control the shape of the noise distributions produced by sampling analog devices. This paper introduces a method for MFVI training using real device noise as the variational distribution. Furthermore, we demonstrate empirically that the predictive distributions from BNN's with the same weight means and variances converge to the same distribution, regardless of the shape of the variational distribution. This result suggests that analog device designers do not need to consider the shape of the device noise distribution when hardware-implementing BNNs performing MFVI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。