研究缺失数据下高维参数估计的统计与计算极限,揭示了部分问题存在可证明的效率瓶颈。
High-dimensional estimation with missing data: Statistical and computational limits
- 在高维正态数据中,针对缺失比例ε的任意缺失机制,分析估计误差与样本量关系
- 均值估计需至少 n ≳ d e^{1/ρ²} 样本才可达到 ρ 误差,而高效算法需 n ≳ d^{1/ρ²} 样本
- 提出基于平方和的多项式时间算法逼近下界,且线性回归中无此类计算瓶颈
我们研究在观测受缺失数据影响时,对总体参数进行计算高效的估计。特别地,考虑真实数据为高斯分布下的可实现污染模型,其中 ε(0 < ε < 1)比例的观测受任意未知的非随机缺失(MNAR)机制影响。当真实数据服从高斯分布时,我们在多个问题中提供了统计-计算间隙的证据。对于 ℓ₂ 范数下的均值估计,为获得不超过 ρ 的误差,几乎需要 n ≳ d e^{1/ρ²} 个样本,且存在一种计算上不高效但能实现该误差的算法。另一方面,任何属于特定主流算法族的计算高效方法均需显著更大的样本复杂度,约为 n ≳ d^{1/ρ²},同时存在一种基于平方和的多项式时间算法几乎达到此下界。在相对算子范数下的协方差估计中也存在类似结果。最后,我们研究含缺失观测的线性回归,发现此类差距不复存在:最小化一个简单强凸经验风险即可在多项式时间内几乎达到信息论下界。
原文摘要 · Abstract (English)
We consider computationally-efficient estimation of population parameters when observations are subject to missing data. In particular, we consider estimation under the realizable contamination model of missing data in which an $ε$ fraction of the observations are subject to an arbitrary (and unknown) missing not at random (MNAR) mechanism. When the true data is Gaussian, we provide evidence towards statistical-computational gaps in several problems. For mean estimation in $\ell_2$ norm, we show that in order to obtain error at most $ρ$, for any constant contamination $ε\in (0, 1)$, (roughly) $n \gtrsim d e^{1/ρ^2}$ samples are necessary and that there is a computationally-inefficient algorithm which achieves this error. On the other hand, we show that any computationally-efficient method within certain popular families of algorithms requires a much larger sample complexity of (roughly) $n \gtrsim d^{1/ρ^2}$ and that there exists a polynomial time algorithm based on sum-of-squares which (nearly) achieves this lower bound. For covariance estimation in relative operator norm, we show that a parallel development holds. Finally, we turn to linear regression with missing observations and show that such a gap does not persist. Indeed, in this setting we show that minimizing a simple, strongly convex empirical risk nearly achieves the information-theoretic lower bound in polynomial time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。