用分段线性正则化实现高效量化，理论保障模型精度与可训练性。
Quantization through Piecewise-Affine Regularization: Optimization and Statistical Guarantees
- 通过连续优化构建分段线性正则框架，解决离散变量优化难题。
- 过参数化下所有临界点高度量化，且能闭式求解各类正则化问题。
- 在回归任务中逼近L1、L2及非凸正则，保持统计保证并输出量化结果。
离散或量化变量的优化问题因搜索空间的组合性质而难以求解。分段线性正则化（PAR）提供了一种灵活的建模与计算框架，基于连续优化实现量化。本文聚焦监督学习场景，从优化与统计角度研究PAR的理论基础：首先，在过参数化情形（参数量大于样本量）下，所有PAR正则损失函数的临界点均表现出高度量化特性；其次，推导出多种（凸、拟凸、非凸）PAR的闭式邻近算子，并展示如何使用邻近梯度法、加速版本及交替方向乘子法求解；最后，研究了PAR正则化线性回归的统计保证，证明可通过PAR逼近经典的ℓ₁、平方ℓ₂及非凸正则形式，获得类似统计性能的同时得到量化解。
原文摘要 · Abstract (English)
Optimization problems over discrete or quantized variables are very challenging in general due to the combinatorial nature of their search space. Piecewise-affine regularization (PAR) provides a flexible modeling and computational framework for quantization based on continuous optimization. In this work, we focus on the setting of supervised learning and investigate the theoretical foundations of PAR from optimization and statistical perspectives. First, we show that in the overparameterized regime, where the number of parameters exceeds the number of samples, every critical point of the PAR-regularized loss function exhibits a high degree of quantization. Second, we derive closed-form proximal mappings for various (convex, quasi-convex, and non-convex) PARs and show how to solve PAR-regularized problems using the proximal gradient method, its accelerated variant, and the Alternating Direction Method of Multipliers. Third, we study statistical guarantees of PAR-regularized linear regression problems; specifically, we can approximate classical formulations of $\ell_1$-, squared $\ell_2$-, and nonconvex regularizations using PAR and obtain similar statistical guarantees with quantized solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。