提出图模型方法,全局求解含非凸正则的参数估计问题
A Graphical Global Optimization Framework for Parameter Estimation of Statistical Models with Nonconvex Regularization Functions
- 用决策图构建原空间强凸松弛,无需额外变量
- 可处理SCAD、MCP等复杂非凸正则,计算验证有效
- 适合需要全局最优解的高维稀疏建模任务
范数约束优化问题广泛存在于投资组合优化、机器学习和特征选择中。传统方法通过拉格朗日松弛将范数约束转化为目标函数中的正则项,但零范数等非凸正则问题求解困难。现有精确方法多引入二值变量和人工边界,转化为高维混合整数规划;或依赖特定结构,难以泛化。替代方法采用具良好统计特性的非凸惩罚,但通常仅用启发式或局部优化。本文提出一种基于图的全局优化框架,可统一处理ℓ_p-范数(p∈[0,∞))及SCAD、MCP等非凸惩罚。利用决策图在原始变量空间构建强凸松弛,避免辅助变量与人工边界。集成于空间分支定界框架,保证收敛至全局最优。在包含复杂非凸惩罚的基准稀疏线性回归问题上进行初步实验,结果表明该方法对现有全局优化技术无法求解的问题具有有效性。
原文摘要 · Abstract (English)
Optimization problems with norm-bounding constraints arise in a variety of applications, including portfolio optimization, machine learning, and feature selection. A common approach to these problems involves relaxing the norm constraint via Lagrangian relaxation, transforming it into a regularization term in the objective function. A particularly challenging class includes the zero-norm function, which promotes sparsity in statistical parameter estimation. Most existing exact methods for solving these problems introduce binary variables and artificial bounds to reformulate them as higher-dimensional mixed-integer programs, solvable by standard solvers. Other exact approaches exploit specific structural properties of the objective, making them difficult to generalize across different problem types. Alternative methods employ nonconvex penalties with favorable statistical characteristics, but these are typically addressed using heuristic or local optimization techniques due to their structural complexity. In this paper, we propose a novel graph-based method to globally solve optimization problems involving generalized norm-bounding constraints. Our approach encompasses standard $\ell_p$-norms for $p \in [0, \infty)$ and nonconvex penalties such as SCAD and MCP. We leverage decision diagrams to construct strong convex relaxations directly in the original variable space, eliminating the need for auxiliary variables or artificial bounds. Integrated into a spatial branch-and-cut framework, our method guarantees convergence to the global optimum. We demonstrate its effectiveness through preliminary computational experiments on benchmark sparse linear regression problems involving complex nonconvex penalties, which are not tractable using existing global optimization techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。