提出高效线性收敛方法,加速稀疏GLM最优性验证
GPU-friendly and Linearly Convergent First-order Methods for Certifying Optimal $k$-sparse GLMs
- 将松弛问题重构为可统一求解的复合优化问题
- 实现对数线性时间计算,比传统方法快数十倍
- 适合需要快速验证稀疏模型最优性的研究者
本文研究稀疏广义线性模型(GLM)的最优性验证问题,通过基数约束实现稀疏性。现有基于分支定界(BnB)框架的方法虽能验证最优性,但其松弛求解过程计算成本高,难以扩展。为此,我们将松弛问题重构成复合优化问题,提出统一的近端框架,具备线性收敛性和计算高效性。在特定几何正则条件下,分析揭示了原始问题二次增长与对偶问题二次衰减的联系,使Fenchel对偶间隙成为解集逼近的精确指标。据此设计基于对偶间隙的重启策略,将广泛子线性近端方法升级为可证明的线性收敛方法,并适用于更广泛场景。针对隐式视角正则项,进一步推导出可在对数线性时间内精确计算正则项及其近端算子的专用算法,避免使用昂贵的通用锥规划求解器。每次迭代主要由矩阵-向量乘法主导,支持GPU加速。在合成及真实数据集上的实验表明,对偶界计算速度提升数量级,大规模实例下的BnB可扩展性显著增强。
原文摘要 · Abstract (English)
We investigate the problem of certifying optimality for sparse generalized linear models (GLMs), where sparsity is enforced through a cardinality constraint. While Branch-and-Bound (BnB) frameworks can certify optimality using perspective relaxations, existing methods for solving these relaxations are computationally intensive, limiting their scalability. To address this challenge, we reformulate the relaxations as composite optimization problems and develop a unified proximal framework that is both linearly convergent and computationally efficient. Under specific geometric regularity conditions, our analysis links primal quadratic growth to dual quadratic decay, yielding error bounds that make the Fenchel duality gap a sharp proxy for progress towards the solution set. This leads to a duality gap-based restart scheme that upgrades a broad class of sublinear proximal methods to provably linearly convergent methods, and applies beyond the sparse GLM setting. For the implicit perspective regularizer, we further derive specialized routines to evaluate the regularizer and its proximal operator exactly in log-linear time, avoiding costly generic conic solvers. The resulting iterations are dominated by matrix--vector multiplications, which enables GPU acceleration. Experiments on synthetic and real-world datasets show orders-of-magnitude faster dual-bound computations and substantially improved BnB scalability on large instances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。