解决未知截断下高维分布参数估计难题,实现多项式时间算法突破。
Efficient Statistics With Unknown Truncation, Polynomial Time Algorithms, Beyond Gaussians
- 提出适用于任意指数族的截断参数估计新方法
- 首次实现高斯分布与线性回归在未知截断下的多项式时间求解
- 适用于半空间或轴对齐矩形截断,适合高维统计建模场景
研究当样本仅在未知集合 $S \\)subseteq \\mathbb{R}^d$ 内才可见时的分布参数估计问题。已有工作表明,在高斯分布且协方差为对角矩阵的情形下,存在 $d^{\mathrm{poly}(1/\varepsilon)}$ 时间算法,但其对 $1/\varepsilon$ 的指数依赖是必要的。本文解决两个开放问题:1)能否扩展到任意高斯分布甚至更广的分布?2)当 $S$ 为简单集如半空间时,能否设计 $\mathrm{poly}(d/\varepsilon)$ 时间算法?我们给出:1)针对满足结构假设的任意指数族,若 $S$ 可被次数为 $\ell$ 的多项式 $\varepsilon$-近似,则存在 $d^{\mathrm{poly}(\ell/\varepsilon)}$ 时间算法;此结果首次实现任意高斯分布和高斯特征线性回归的未知截断参数估计。2)当 $S$ 为半空间或轴对齐矩形时,对包含所有高斯分布的指数族类,可实现 $\mathrm{poly}(d/\varepsilon)$ 时间算法。途中还发展了对协变量偏移鲁棒的正负样本学习归约工具。
原文摘要 · Abstract (English)
We study the estimation of distributional parameters when samples are shown only if they fall in some unknown set $S \subseteq \mathbb{R}^d$. Kontonis, Tzamos, and Zampetakis (FOCS'19) gave a $d^{\mathrm{poly}(1/\varepsilon)}$ time algorithm for finding $\varepsilon$-accurate parameters for the special case of Gaussian distributions with diagonal covariance matrix. Recently, Diakonikolas, Kane, Pittas, and Zarifis (COLT'24) showed that this exponential dependence on $1/\varepsilon$ is necessary even when $S$ belongs to some well-behaved classes. These works leave the following open problems which we address in this work: Can we estimate the parameters of any Gaussian or even extend beyond Gaussians? Can we design $\mathrm{poly}(d/\varepsilon)$ time algorithms when $S$ is a simple set such as a halfspace? We make progress on both of these questions by providing the following results: 1. Toward the first question, we give a $d^{\mathrm{poly}(\ell/\varepsilon)}$ time algorithm for any exponential family that satisfies some structural assumptions and any unknown set $S$ that is $\varepsilon$-approximable by degree-$\ell$ polynomials. This result has two important applications: 1a) The first algorithm for estimating arbitrary Gaussian distributions from samples truncated to an unknown $S$; and 1b) The first algorithm for linear regression with unknown truncation and Gaussian features. 2. To address the second question, we provide an algorithm with runtime $\mathrm{poly}(d/\varepsilon)$ that works for a set of exponential families (containing all Gaussians) when $S$ is a halfspace or an axis-aligned rectangle. Along the way, we develop tools that may be of independent interest, including, a reduction from PAC learning with positive and unlabeled samples to PAC learning with positive and negative samples that is robust to certain covariate shifts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。