用耦合自由能优化变分推断，让模型更抗异常值且训练更稳定。
Variational Inference Optimized Using the Curved Geometry of Coupled Free Energy
- 基于耦合自由能构建新优化框架，适配重尾分布的弯曲几何结构。
- 在CelebA上训练5轮后，重构图像的Wasserstein-2距离比VAE提升3%。
- 适合需要鲁棒建模和稳定训练的复杂分布建模任务。
我们提出一种基于耦合自由能的变分推断优化框架，将变分推断推广至耦合指数族，该族包含广义帕累托和学生t等重要重尾分布。通过利用耦合自由能（即反概率的耦合证据下界）改进模型精度与鲁棒性。引入耦合的Fisher信息度量与仿射联络。该方法用于设计耦合变分自编码器（CVAE），通过同时对分布与损失函数进行耦合，重构损失仍为均方平均误差，仅常数项调整。创新在于使用关联耦合概率采样重尾潜在变量，其尾部衰减更快。结果是模型能有效抵御严重异常值，同时保证训练稳定性。在CelebA数据集上，5轮训练后重构图像的Wasserstein-2或Fréchet inception距离相比标准VAE提升3%。
原文摘要 · Abstract (English)
We introduce an optimization framework for variational inference based on the coupled free energy, extending variational inference techniques to account for the curved geometry of the coupled exponential family. This family includes important heavy-tailed distributions such as the generalized Pareto and the Student's t. By leveraging the coupled free energy, which is equal to the coupled evidence lower bound (ELBO) of the inverted probabilities, we improve the accuracy and robustness of the learned model. The coupled generalization of Fisher Information metric and the affine connection. The method is applied to the design of a coupled variational autoencoder (CVAE). By using the coupling for both the distributions and cost functions, the reconstruction metric is derived to still be the mean-square average loss with modified constants. The novelty comes from sampling the heavy-tailed latent distribution with its associated coupled probability, which has faster decaying tails. The result is the ability to train a model robust against severe outliers, while assuring that the training process is stable. The Wasserstein-2 or Fréchet Inception Distance of the reconstructed CelebA images shows the CVAE has a 3\% improvement over the VAE after 5 epochs of training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。