用路径传输方法提升深度高斯过程推理效率,效果优于传统方法。
Onsager-Machlup Posterior Transport for Deep Gaussian Processes
- 将诱导变量推断转化为路径传输问题,利用正则化确定性采样器映射参考分布。
- 在两个大尺寸数据集上显著降低负对数似然和均方误差,性能超越现有方法。
- 适合追求高效精确推理的机器学习研究者,尤其在大规模数据场景下优势明显。
深度高斯过程(DGPs)中近似诱导变量推断是核心计算瓶颈。现有方法或通过变分推断拟合显式密度(如DSVI、IPVI、DDVI、DBVI),或采用马尔可夫链蒙特卡洛采样(如SGHMC)。本文提出一种后验路径传输框架:学习一个确定性采样器,将可处理的参考测度映射到后验相关的诱导变量,路径正则项基于杜布桥接参考扩散导出的路径先验。所提出的OM-Path(形式上为FBVI-bridge-Path)利用Song的概率流微分方程,应用于DBVI中的杜布桥接前向随机微分方程;参考漂移由桥接边缘系数解析给出(无需得分匹配),路径正则项为奥恩斯格-马赫卢普作用量。在训练时使用的有限ε值下,目标函数等价于温化杜布桥路径后验的未归一化对数密度,定理1表明其与小噪声条件下最大后验路径一致。基于相同桥接结构,推导出两种严格的路径空间ELBO变体(FFJORD行列式项;OM正则化连续时间网络)。在七个UCI回归基准上与DBVI进行配对威尔科克森检验,OM-Path在两个最大数据集上取得统计显著优势:'power'数据集(p=0.014,NLL=0.012,接近DSVI基线0.017);'protein'数据集(p=0.002,RMSE=0.716 vs. 0.764,NLL=1.086 vs. 1.149),在'yacht'/'qsar'上持平,但在小样本高噪声数据的'boston'/'energy'/'concrete'上落后于DBVI。严格ELBO变体在任一指标上均未超越DBVI:在此设置下,路径目标方差最小化比精确密度追踪更重要。
原文摘要 · Abstract (English)
Approximate inference over inducing variables is the central computational bottleneck of Deep Gaussian Processes (DGPs). Existing methods either fit an explicit density $q_ϕ(\bU)$ by an ELBO (DSVI, IPVI, DDVI, DBVI) or sample by MCMC (SGHMC). We instead frame DGP inference as \emph{posterior transport}: learn a deterministic sampler that maps a tractable reference measure to posterior-relevant inducing variables, regularised by a path prior derived from the Doob-bridged reference diffusion. Our realisation, \textbf{OM-Path} (formally FBVI-bridge-Path), uses Song's probability-flow ODE applied to DBVI's Doob-bridged forward SDE; the reference drift is closed-form from the bridge marginal coefficients (no score matching) and the path regulariser is the \textbf{Onsager--Machlup action}. At the finite-$ε$ value used at training, the objective is the negative log unnormalised density of a tempered Doob-bridge path posterior, and Theorem 1 identifies it with the same posterior's small-noise MAP path via the Freidlin--Wentzell LDP. Two strict path-space ELBO variants on the same bridge backbone (FFJORD log-det; OM-regularised CNF) are derived as ablations. Under a matched-seed paired Wilcoxon test against DBVI on seven UCI regression benchmarks, OM-Path delivers statistically significant wins on the two largest datasets (\textit{power}: $p\!=\!0.014$, NLL $\mathbf{0.012}$ matching the DSVI baseline of $0.017$; \textit{protein}: $p\!=\!0.002$, RMSE $\mathbf{0.716}$ vs.\ $0.764$, NLL $\mathbf{1.086}$ vs.\ $1.149$), statistical ties on \textit{yacht} / \textit{qsar}, and concedes \textit{boston} / \textit{energy} / \textit{concrete} to DBVI on small-$N$ noisy data. The strict-ELBO variants do not clear DBVI on any UCI metric: in this regime, reducing the variance of the path objective dominates exact-density tracking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。