arXiv:2411.12700cs.LGcs.DS2024-11ICML被引 4

用不准确的先验信息,让高维高斯分布学习更省样本。

Learning multivariate Gaussians with imperfect advice

论文配图:Learning multivariate Gaussians with imperfect advice
图 1 · 摘自论文原文
  • 利用有误差的协方差矩阵作为先验,动态调整学习样本量。
  • 当先验误差满足特定条件时,样本复杂度可降至 $\tilde{O}(d^{2-β}/\varepsilon^2)$。
  • 适合对样本效率敏感的高维分布学习场景,如金融或生物数据建模。

我们重新审视学习增强算法框架下的分布学习问题。在该框架中,给定一个可能不准确的关于真实未知分布的概率分布作为建议。目标是设计学习算法,使其样本复杂度随建议质量提升而降低,从而在建议足够准确时突破经典学习下界。具体而言,我们在PAC学习设置下证明了多变量高斯分布 $N(\boldsymbolμ, \boldsymbolΣ)$ 的学习可实现此目标。经典无建议情形下,$\tildeΘ(d^2/\varepsilon^2)$ 个样本足以以常数概率在总变差距离 $\varepsilon$ 内学习 $d$ 维高斯分布。当额外提供协方差估计 $\tilde{\boldsymbolΣ}$ 作为建议时,若满足 $\| \tilde{\boldsymbolΣ}^{-1/2} \boldsymbolΣ \tilde{\boldsymbolΣ}^{-1/2} - \boldsymbol{I_d} \|_1 \leq \varepsilon d^{1-β}$($\|\cdot\|_1$ 为逐元素 $\ell_1$ 范数),则 $\tilde{O}(d^{2-β}/\varepsilon^2)$ 个样本即可完成学习,实现多项式级改进。

原文摘要 · Abstract (English)

We revisit the problem of distribution learning within the framework of learning-augmented algorithms. In this setting, we explore the scenario where a probability distribution is provided as potentially inaccurate advice on the true, unknown distribution. Our objective is to develop learning algorithms whose sample complexity decreases as the quality of the advice improves, thereby surpassing standard learning lower bounds when the advice is sufficiently accurate. Specifically, we demonstrate that this outcome is achievable for the problem of learning a multivariate Gaussian distribution $N(\boldsymbolμ, \boldsymbolΣ)$ in the PAC learning setting. Classically, in the advice-free setting, $\tildeΘ(d^2/\varepsilon^2)$ samples are sufficient and worst case necessary to learn $d$-dimensional Gaussians up to TV distance $\varepsilon$ with constant probability. When we are additionally given a parameter $\tilde{\boldsymbolΣ}$ as advice, we show that $\tilde{O}(d^{2-β}/\varepsilon^2)$ samples suffices whenever $\| \tilde{\boldsymbolΣ}^{-1/2} \boldsymbolΣ \tilde{\boldsymbolΣ}^{-1/2} - \boldsymbol{I_d} \|_1 \leq \varepsilon d^{1-β}$ (where $\|\cdot\|_1$ denotes the entrywise $\ell_1$ norm) for any $β> 0$, yielding a polynomial improvement over the advice-free setting.

分布学习高斯分布学习增强样本效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。