提出新方法，让带负系数混合模型在高维下快速估算期望值。
Scalable Expectation Estimation with Subtractive Mixture Models
- 用差分表示法构建无偏重要性采样估计器，无需直接采样负系数模型。
- 在高维场景下，速度远超自回归采样，且估计精度相当。
- 适合需要高效高维期望估计的研究者，尤其关注生成建模与采样优化。
许多蒙特卡洛（MC）和重要性采样（IS）方法使用混合模型（MM）因其结构简单且能捕捉多峰分布。最近，带有负系数的减法混合模型（SMM）在生成建模中表现出更强的表达能力。然而，其负参数使采样复杂化，需依赖昂贵的自回归技术或接受-拒绝算法，难以在高维场景下扩展。本文利用SMM的差分表示，构造了一个无偏的重要性采样估计器（ΔEx），无需对SMM进行采样，从而实现高维期望估计。实验表明，ΔEx在蒙特卡洛估计中可达到与自回归采样相当的精度，同时显著更快。此外，我们通过手工设计提议分布进行了初步实验，首次获得关于如何为ΔEx构建安全提议的洞察。
原文摘要 · Abstract (English)
Many Monte Carlo (MC) and importance sampling (IS) methods use mixture models (MMs) for their simplicity and ability to capture multimodal distributions. Recently, subtractive mixture models (SMMs), i.e. MMs with negative coefficients, have shown greater expressiveness and success in generative modeling. However, their negative parameters complicate sampling, requiring costly auto-regressive techniques or accept-reject algorithms that do not scale in high dimensions. In this work, we use the difference representation of SMMs to construct an unbiased IS estimator ($Δ\text{Ex}$) that removes the need to sample from the SMM, enabling high-dimensional expectation estimation with SMMs. In our experiments, we show that $Δ\text{Ex}$ can achieve comparable estimation quality to auto-regressive sampling while being considerably faster in MC estimation. Moreover, we conduct initial experiments with $Δ\text{Ex}$ using hand-crafted proposals, gaining first insights into how to construct safe proposals for $Δ\text{Ex}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。