构建可深度堆叠的贝叶斯模型,实现闭式推断与不确定性校准。
Composing Non-Conjugate Factor Graphs with Closed-Form Variational Inference

- 用五类基础因子图构件组合模型,保持闭式变分推断
- 深度路由层可编码任意决策树,支持函数万能逼近
- 用于时间序列预测,自动推断专家选择,提供校准不确定性
将概率模块堆叠为深层架构通常破坏闭式推断。本文证明闭式推断可被保留:识别出五种因子图基本单元——双线性因子、指数链接、伽马先验、高斯似然和等式节点,并证明由它们构成的任何模型均支持闭式变分消息传递。该方法有效是因为每类单元仅保持少量消息分布族:在均值场分解下,高斯变量的消息保持高斯分布,精度变量的消息保持伽马分布;非共轭接口(指数链接)通过高斯矩生成函数和伽马族充分统计量仍可计算。我们演示了从静态集成到输入依赖门控、再到分支路由的逐层组合,表明堆叠路由层可编码任意决策树,从而实现具有闭式推断的万能函数逼近。应用于集成时间序列预测,该框架形成贝叶斯专家混合模型,其中门控函数通过推断而非学习获得,在五个基准数据集上提供对专家选择的校准不确定性估计。
原文摘要 · Abstract (English)
Stacking probabilistic building blocks into deeper architectures typically breaks closed-form inference. We show that closed-form inference can be preserved. We identify five factor-graph primitives: a bilinear factor, an exponential link, a Gamma prior, a Gaussian likelihood, and an equality node, and prove that any model composed from them admits closed-form variational message passing. The construction works because each primitive preserves a small set of message families: under mean-field factorization, messages on Gaussian variables remain Gaussian and messages on precision variables remain Gamma, while the only non-conjugate interface, the exponential link, remains tractable through the Gaussian moment-generating function and the sufficient statistics of the Gamma family. We demonstrate composition at increasing depth, from static ensembles through input-dependent gating to split-branch routing, and show that stacking routing layers encodes arbitrary decision trees, establishing universal function approximation with closed-form inference. Applied to ensemble time-series forecasting, the framework yields a Bayesian mixture of experts in which gating functions are inferred rather than learned, providing calibrated uncertainty over expert selection across five benchmark datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。