用可能性理论重构变分推断,提升不确定信息下的模型鲁棒性。
Maxitive Donsker-Varadhan Formulation for Possibilistic Variational Inference
- 提出最大值型多斯克-瓦拉达汉公式,替代传统期望积分
- 设计基于指数族的更新规则,在图像分类中表现媲美主流方法
- 适合处理数据稀疏或信息不精确的场景,提升模型可解释性
变分推断(VI)是现代贝叶斯学习的核心,可在复杂模型中实现近似推理。然而其依赖高维积分定义的期望与散度,常导致解析求解困难,需大量近似。可能性理论是一种非精确概率框架,可直接建模认知不确定性,无需主观概率解释。尽管该框架在稀疏或不精确信息下具备鲁棒性与可解释性,但将VI适配至可能性设定需重新思考核心概念如散度——后者依赖可加性。本文通过建立经典多斯克-瓦拉达汉公式的最大值类比,提出一种原则性可能论变分推断框架。该框架可导出基于指数族候选分布的学习规则,并给出神经网络训练的实用更新规则,形成一类优化器,称为CBOpt。实验表明,CBOpt在域内与域外图像分类任务中均达到具有竞争力的性能。
原文摘要 · Abstract (English)
Variational inference (VI) is a cornerstone of modern Bayesian learning, enabling approximate inference in complex models. However, its formulation depends on expectations and divergences defined through high-dimensional integrals, often rendering analytical treatment impossible and necessitating heavy reliance on approximations. Possibility theory, an imprecise probability framework, allows us to directly model epistemic uncertainty instead of relying on a subjective interpretation of probabilities. While this framework provides robustness and interpretability under sparse or imprecise information, adapting VI to the possibilistic setting requires rethinking core concepts such as divergences, which presuppose additivity. In this work, we develop a principled formulation for performing possibilistic VI by establishing a maxitive analogue of the classical Donsker-Varadhan formulation. The resulting framework enables us to derive a learning rule for possibilistic VI with exponential-family candidates and practical update rules for neural-network training, giving rise to a family of optimizers termed CBOpt. Finally, we demonstrate that CBOpt achieves competitive performance on both in-domain and out-of-domain image classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。