arXiv:2606.01954cs.LGstat.ML2026-06

用流模型提升函数空间变分推断的表达能力,更好捕捉复杂后验分布。

Flow-Transformed Implicit Processes for Function-Space Variational Inference

论文配图:Flow-Transformed Implicit Processes for Function-Space Variational Inference
图 1 · 摘自论文原文
  • 用归一化流替代高斯分布,增强组合权重的表达力
  • 在函数空间中成功建模非对称、多峰后验结构
  • 适合需要精细不确定性建模的贝叶斯机器学习任务

隐式过程先验通过灵活生成机制定义函数分布,适用于贝叶斯函数空间建模。然而,由于其诱导的函数空间分布通常无闭式解,后验推断极具挑战。现有方法常通过采样函数构建有限近似,并将后验函数表示为这些样本的加权组合,但通常采用高斯变分分布来建模组合权重,限制了后验不确定性的表达能力,尤其当真实后验为非对称、重尾或多重模式时。本文提出流变换隐式过程(FTIP),一种更具表达力的变分推断方法。不同于传统高斯权重分布,FTIP使用归一化流定义更丰富的变分分布,从而在保持优化可处理性的同时,实现灵活的函数空间后验建模。模型采用黑箱α目标进行训练,支持质量覆盖与模式聚焦的变分行为对比。实验表明,相比高斯系数近似,FTIP能有效捕捉函数空间中的非对称与多峰后验结构,避免平滑或坍缩。

原文摘要 · Abstract (English)

Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space modelling. However, performing posterior inference with such priors is challenging because their induced function-space distributions are typically not available in closed form. One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples. Existing approaches commonly place a Gaussian variational distribution over the combination weights. While tractable, this choice limits the shapes of posterior uncertainty that can be represented, especially when the true posterior is asymmetric, heavy-tailed, or multimodal. We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive. Instead of using a Gaussian distribution over the combination weights, FTIP uses a normalizing flow to define a richer variational distribution. This induces a flexible posterior distribution over functions while preserving tractable optimization. We train the model using a Black-Box α objective, allowing us to compare mass-covering and mode-seeking variational behaviour. Experiments show that FTIP captures asymmetric and multimodal posterior structure in function space that Gaussian coefficient approximations tend to smooth or collapse.

变分推断函数空间归一化流贝叶斯建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。