针对稀疏向量的降维,提出更优的平均情况保证方法。
Beyond Worst-Case Dimensionality Reduction for Sparse Vectors
- 基于非负稀疏性设计非线性映射,实现更小维度降维。
- 在任意ℓ_p范数下保持距离精度1±ε,维度仅依赖于稀疏度和数据量。
- 适用于需要高精度距离保留的推荐系统、聚类等场景。
本文研究稀疏向量的超越最坏情况维度压缩。首先考虑平均情况保证:对于任意s-稀疏向量集合X⊆ℝ^d,存在线性映射至ℝ^{O(s²)}可精确保留99%向量的ℓ_p范数。我们证明该维度下界为Ω(s²),对平滑映射(如编码器-解码器)同样成立。若允许非光滑函数(如压缩感知),则可降至O(s log d)。进一步,在非负稀疏向量假设下,可非线性嵌入至O(s log(|X|s)/ε²)维,以1±ε精度保留所有成对ℓ_p距离,且不依赖p。此外,ℓ_∞范数下可精确降维至O(s log |X|),且紧致。非线性与非负性均不可省略,并带来下游算法改进。
原文摘要 · Abstract (English)
We study beyond worst-case dimensionality reduction for $s$-sparse vectors. Our work is divided into two parts, each focusing on a different facet of beyond worst-case analysis: We first consider average-case guarantees. A folklore upper bound based on the birthday-paradox states: For any collection $X$ of $s$-sparse vectors in $\mathbb{R}^d$, there exists a linear map to $\mathbb{R}^{O(s^2)}$ which \emph{exactly} preserves the norm of $99\%$ of the vectors in $X$ in any $\ell_p$ norm (as opposed to the usual setting where guarantees hold for all vectors). We give lower bounds showing that this is indeed optimal in many settings: any oblivious linear map satisfying similar average-case guarantees must map to $Ω(s^2)$ dimensions. The same lower bound also holds for a wide class of smooth maps, including `encoder-decoder schemes', where we compare the norm of the original vector to that of a smooth function of the embedding. These lower bounds reveal a separation result, as an upper bound of $O(s \log(d))$ is possible if we instead use arbitrary (possibly non-smooth) functions, e.g., via compressed sensing algorithms. Given these lower bounds, we specialize to sparse \emph{non-negative} vectors. For a dataset $X$ of non-negative $s$-sparse vectors and any $p \ge 1$, we can non-linearly embed $X$ to $O(s\log(|X|s)/ε^2)$ dimensions while preserving all pairwise distances in $\ell_p$ norm up to $1\pm ε$, with no dependence on $p$. Surprisingly, the non-negativity assumption enables much smaller embeddings than arbitrary sparse vectors, where the best known bounds suffer exponential dependence. Our map also guarantees \emph{exact} dimensionality reduction for $\ell_{\infty}$ by embedding into $O(s\log |X|)$ dimensions, which is tight. We show that both the non-linearity of $f$ and the non-negativity of $X$ are necessary, and provide downstream algorithmic improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。