提出高效一阶算法,加速大规模稀疏GLM的最优性验证
Scalable First-order Method for Certifying Optimal k-Sparse GLMs
- 用近似松弛问题的梯度算法替代复杂二阶规划
- 对偶界计算速度提升,实测可处理上万变量问题
- 适合需要严格证明解最优的大规模稀疏建模场景
本文研究稀疏广义线性模型(GLM)的最优性认证问题,其中稀疏性通过ℓ₀范数约束实现。尽管分支定界(BnB)框架可通过对偶界剪枝节点来认证最优性,但现有对偶界计算方法或计算成本高,或收敛慢,难以扩展至大规模问题。为此,我们提出一种一阶近端梯度算法,用于在BnB框架内求解问题的视角松弛。具体地,将松弛问题表述为复合优化问题,并证明其非光滑部分的近端算子可在对数线性时间复杂度内精确计算,无需求解代价高昂的二阶锥规划。此外,引入简单重启策略,在保持每轮低复杂度的同时显著提升收敛速度。在合成与真实数据集上的大量实验表明,该方法显著加速对偶界计算,对大规模问题具有极强的最优性认证能力。
原文摘要 · Abstract (English)
This paper investigates the problem of certifying optimality for sparse generalized linear models (GLMs), where sparsity is enforced through an $\ell_0$ cardinality constraint. While branch-and-bound (BnB) frameworks can certify optimality by pruning nodes using dual bounds, existing methods for computing these bounds are either computationally intensive or exhibit slow convergence, limiting their scalability to large-scale problems. To address this challenge, we propose a first-order proximal gradient algorithm designed to solve the perspective relaxation of the problem within a BnB framework. Specifically, we formulate the relaxed problem as a composite optimization problem and demonstrate that the proximal operator of the non-smooth component can be computed exactly in log-linear time complexity, eliminating the need to solve a computationally expensive second-order cone program. Furthermore, we introduce a simple restart strategy that enhances convergence speed while maintaining low per-iteration complexity. Extensive experiments on synthetic and real-world datasets show that our approach significantly accelerates dual bound computations and is highly effective in providing optimality certificates for large-scale problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。