用减法混合模型提升推断近似效果，解决传统方法表达能力不足问题。
How to Approximate Inference with Subtractive Mixture Models
- 设计新期望估计器与学习方案，让带负系数混合模型适用于变分推断和重要性采样
- 实验证明其在分布逼近任务中优于经典混合模型，提升近似精度
- 针对数值不稳与训练效率问题提出改进策略，适合需要高精度推断的研究者
经典混合模型广泛用于变分推断（VI）和重要性采样（IS）等近似推断场景。近期提出的带负系数混合模型（减法混合模型，SMMs）被认为具有更强的表达能力，但因缺乏隐变量语义，无法直接采用传统混合模型的采样方法。本文研究如何克服这一难题，设计了适用于IS的若干期望估计器以及用于VI的学习方案，并在分布逼近任务上进行了实证评估。此外，还讨论了由此带来的估计稳定性与学习效率挑战，并提出相应解决方案。代码已公开于：https://github.com/april-tools/delta-vi。
原文摘要 · Abstract (English)
Classical mixture models (MMs) are widely used tractable proposals for approximate inference settings such as variational inference (VI) and importance sampling (IS). Recently, mixture models with negative coefficients, called subtractive mixture models (SMMs), have been proposed as a potentially more expressive alternative. However, how to effectively use SMMs for VI and IS is still an open question as they do not provide latent variable semantics and therefore cannot use sampling schemes for classical MMs. In this work, we study how to circumvent this issue by designing several expectation estimators for IS and learning schemes for VI with SMMs, and we empirically evaluate them for distribution approximation. Finally, we discuss the additional challenges in estimation stability and learning efficiency that they carry and propose ways to overcome them. Code is available at: https://github.com/april-tools/delta-vi.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。