提出一种概率迭代硬阈值方法，解决高维数据中稀疏性优化的收敛难题。
Probabilistic Iterative Hard Thresholding for Sparse Learning
- 基于概率机制的迭代硬阈值，处理带基数约束的期望优化问题
- 理论证明该随机过程在噪声梯度下仍可收敛
- 适用于高维稀疏学习，适合做变量选择或模型压缩的研究者
当数据维度远高于样本量时，发现真实模型中的隐藏稀疏性对于构建准确统计模型至关重要。所谓的“l0范数”通过计数向量中非零元素数量，是强化稀疏性的可靠机制，若将其纳入优化问题以最小化模型对观测数据的拟合误差，效果显著。然而，在大数据场景下，由于计算需求必须使用噪声梯度估计，现有文献中缺乏能可靠收敛的方法。本文提出一种求解带有基数约束的期望目标优化问题的新方法，证明了其底层随机过程的收敛性，并在两个机器学习任务上展示了性能表现。
原文摘要 · Abstract (English)
For statistical modeling wherein the data regime is unfavorable in terms of dimensionality relative to the sample size, finding hidden sparsity in the ground truth can be critical in formulating an accurate statistical model. The so-called "l0 norm" which counts the number of non-zero components in a vector, is a strong reliable mechanism of enforcing sparsity when incorporated into an optimization problem for minimizing the fit of a given model to a set of observations. However, in big data settings wherein noisy estimates of the gradient must be evaluated out of computational necessity, the literature is scant on methods that reliably converge. In this paper we present an approach towards solving expectation objective optimization problems with cardinality constraints. We prove convergence of the underlying stochastic process, and demonstrate the performance on two Machine Learning problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。