提出一种新算法,可在复杂约束下实现稀疏优化的全局最优保证。
Optimization over Sparse Support-Preserving Sets: Two-Step Projection with Global Optimality Guarantees
- 采用两步投影机制替代传统投影,解决混合约束下的稀疏优化问题。
- 在确定性、随机与零阶设置下均给出目标值的全局最优保证。
- 改进经典三点引理,适用于非凸投影,适合需严格稀疏控制的研究者。
在稀疏优化中,使用ℓ₀伪范数施加硬约束可实现可控稀疏性,优于凸松弛方法。然而,许多实际应用不仅需要稀疏性约束,还需满足额外条件。现有算法通常依赖于混合约束的闭式投影(可能不存在),或仅提供局部收敛保证,难以满足稀疏优化中对全局最优的期望。本文研究文献中常见的支持保持型附加约束下的稀疏优化问题。提出一种改进的迭代硬阈值算法,配备专为混合约束设计的两步连续投影算子,作为欧氏投影的简单替代。通过引入稀疏性松弛与次优性的新权衡,我们在常规的限制强凸性/光滑性假设下,于确定性、随机及零阶设置中提供了输出目标值的全局保证。在证明技术上,我们首次将经典三点引理推广至所考虑的两步非凸投影算子,从而以优雅方式分析目标值收敛性,这是以往方法无法实现的。在零阶情形下,该技术甚至优于de Vazelhes等(2022)的最先进结果,即便在无额外约束时也可消除其工作中存在的非消失系统误差。
原文摘要 · Abstract (English)
In sparse optimization, enforcing hard constraints using the $\ell_0$ pseudo-norm offers advantages like controlled sparsity compared to convex relaxations. However, many real-world applications demand not only sparsity constraints but also some extra constraints. While prior algorithms have been developed to address this complex scenario with mixed combinatorial and convex constraints, they typically require the closed form projection onto the mixed constraints which might not exist, and/or only provide local guarantees of convergence which is different from the global guarantees commonly sought in sparse optimization. To fill this gap, in this paper, we study the problem of sparse optimization with extra support-preserving constraints commonly encountered in the literature. We present a new variant of iterative hard-thresholding algorithm equipped with a two-step consecutive projection operator customized for these mixed constraints, serving as a simple alternative to the Euclidean projection onto the mixed constraint. By introducing a novel trade-off between sparsity relaxation and sub-optimality, we provide global guarantees in objective value for the output of our algorithm, in the deterministic, stochastic, and zeroth-order settings, under the conventional restricted strong-convexity/smoothness assumptions. As a fundamental contribution in proof techniques, we develop a novel extension of the classic three-point lemma to the considered two-step non-convex projection operator, which allows us to analyze the convergence in objective value in an elegant way that has not been possible with existing techniques. In the zeroth-order case, such technique also improves upon the state-of-the-art result from de Vazelhes et. al. (2022), even in the case without additional constraints, by allowing us to remove a non-vanishing system error present in their work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。