用物理电路模拟贝叶斯推断,实现快速低耗的不确定性计算。
Thermodynamic Bayesian Inference
- 用热力学电路直接实现朗之万采样,物理层面逼近后验分布。
- 高维模型采样时间仅随维度对数增长,能量开销为 d·ln(d)。
- 适合需要高效贝叶斯推断的硬件加速场景,如边缘智能与可靠性系统。
对复杂预测模型(如深度神经网络)进行完整的贝叶斯处理,可实现严格的不确定性量化,并自动化模型选择等高层任务。然而,由于在多参数空间中采样贝叶斯后验的不可行性,贝叶斯方法在最需要的地方难以应用。热力学计算作为一种加速机器学习运算(如矩阵求逆)的新范式,基于将朗之万方程映射到含噪物理系统的动力学。因此,自然考虑在热力学设备上实现朗之万采样算法。本文提出电子类比器件,通过物理实现朗之万动力学来采样贝叶斯后验。给出了高斯-高斯模型和贝叶斯逻辑回归的电路设计,并通过仿真验证。在合理假设下,这些模型的后验采样时间与维度的对数 #ln(d)# 成正比,其中 $d$ 为维度。对于高斯-高斯模型,能量开销与 $d \ln(d)$ 成正比。结果表明,热力学计算具有实现高速、低能耗贝叶斯推断的巨大潜力。
原文摘要 · Abstract (English)
A fully Bayesian treatment of complicated predictive models (such as deep neural networks) would enable rigorous uncertainty quantification and the automation of higher-level tasks including model selection. However, the intractability of sampling Bayesian posteriors over many parameters inhibits the use of Bayesian methods where they are most needed. Thermodynamic computing has emerged as a paradigm for accelerating operations used in machine learning, such as matrix inversion, and is based on the mapping of Langevin equations to the dynamics of noisy physical systems. Hence, it is natural to consider the implementation of Langevin sampling algorithms on thermodynamic devices. In this work we propose electronic analog devices that sample from Bayesian posteriors by realizing Langevin dynamics physically. Circuit designs are given for sampling the posterior of a Gaussian-Gaussian model and for Bayesian logistic regression, and are validated by simulations. It is shown, under reasonable assumptions, that the Bayesian posteriors for these models can be sampled in time scaling with $\ln(d)$, where $d$ is dimension. For the Gaussian-Gaussian model, the energy cost is shown to scale with $ d \ln(d)$. These results highlight the potential for fast, energy-efficient Bayesian inference using thermodynamic computing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。