密度估计中,样本与查询时间难以同时显著减少。
Statistical-Computational Trade-offs for Density Estimation
- 提出统计-计算权衡下界,揭示样本与查询时间的固有矛盾。
- 在多项式空间下,样本少则查询时间接近线性,反之亦然。
- 适用于均匀分布等典型场景,对实际算法设计有指导意义。
我们研究了如下密度估计问题:给定 $k$ 个定义在离散域 $[n]$ 上的分布 $p_1, dots, p_k$,以及从某个查询分布 $q$ 采样的数据,目标是输出一个与 $q$ 接近的 $p_i$。近期工作~\cite{aamand2023data} 首次实现了在采样复杂度和查询时间上均低于线性的结果,且保持多项式空间。但其改进仅是亚多项式因子。本文的主要成果是证明:对于一大类数据结构,该结果无法被显著提升。具体而言,若算法使用 $O(n / \log^c k)$ 样本($c>0$ 为常数)并保持多项式空间,则查询时间至少为 $k^{1 - O(1)/\log \log k}$,即接近于 $k$ 的线性。这一结果建立了密度估计中的新型统计-计算权衡,表明任何数据结构必须在样本数量或查询时间上接近线性。该下界在可实现情形(即 $q = p_i$)及分布为半域均匀时依然成立。此外,我们还构造了一个简单数据结构,其上界渐近匹配下界,实验显示其在实践中效率较高。
原文摘要 · Abstract (English)
We study the density estimation problem defined as follows: given $k$ distributions $p_1, \ldots, p_k$ over a discrete domain $[n]$, as well as a collection of samples chosen from a ``query'' distribution $q$ over $[n]$, output $p_i$ that is ``close'' to $q$. Recently~\cite{aamand2023data} gave the first and only known result that achieves sublinear bounds in {\em both} the sampling complexity and the query time while preserving polynomial data structure space. However, their improvement over linear samples and time is only by subpolynomial factors. Our main result is a lower bound showing that, for a broad class of data structures, their bounds cannot be significantly improved. In particular, if an algorithm uses $O(n/\log^c k)$ samples for some constant $c>0$ and polynomial space, then the query time of the data structure must be at least $k^{1-O(1)/\log \log k}$, i.e., close to linear in the number of distributions $k$. This is a novel \emph{statistical-computational} trade-off for density estimation, demonstrating that any data structure must use close to a linear number of samples or take close to linear query time. The lower bound holds even in the realizable case where $q=p_i$ for some $i$, and when the distributions are flat (specifically, all distributions are uniform over half of the domain $[n]$). We also give a simple data structure for our lower bound instance with asymptotically matching upper bounds. Experiments show that the data structure is quite efficient in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。