提出稀疏二次支持向量机,解决高维非线性分类过拟合问题。
L0-Regularized Quadratic Surface Support Vector Machines
- 用L0正则约束参数数量,实现模型稀疏化
- 在多个数据集上达到与传统SVM相当的准确率
- 适合需要可解释性的金融信用评分场景
无核二次曲面支持向量机(QSVM)因其无需核函数即可灵活建模非线性决策边界而受到关注。然而,全二次分类器使参数量随数据维度平方增长,易导致过拟合且难以解释。为此,我们通过施加参数个数约束,提出稀疏型QSVM。虽然引入L0范数会增加计算复杂度,但我们设计了一种惩罚分解算法,可生成满足一阶Lu-Zhang最优条件的解。算法中的子问题或有闭式解,或可通过对偶形式高效求解,保障整体效率。我们还分析了该算法在两种损失设置下的收敛性。在公开基准数据集上的数值实验表明,所提模型性能优于常见SVM变体,并产生预期的稀疏解;其在真实信贷数据集上的优异表现,显示其在信用评分中的应用潜力。
原文摘要 · Abstract (English)
Kernel-free quadratic surface support vector machines (QSVM) have recently gained traction due to their flexibility in modeling nonlinear decision boundaries without relying on kernel functions. However, the introduction of a full quadratic classifier significantly increases the number of model parameters, scaling quadratically with data dimensionality, which often leads to overfitting and makes interpretation difficult. To address these challenges, we propose sparse variants of the QSVM by enforcing a cardinality constraint on the model parameters. While enhancing generalization and promoting sparsity, leveraging the $\ell_0$-norm inevitably incurs additional computational complexity. To tackle this, we develop a penalty decomposition algorithm capable of producing solutions that provably satisfy the first-order Lu-Zhang optimality conditions. We show that the subproblems arising within the algorithm either admit closed-form solutions or can be solved efficiently through dual formulations, which contributes to the method's overall effectiveness. Besides, we analyze the convergence behavior of the algorithm under both loss settings. In addition, the numerical experiments on public benchmark datasets indicate that the proposed model is competitive with commonly used SVM variants and produces sparse solutions as expected. Moreover, its strong performance on real-world credit datasets demonstrates its potential for credit scoring applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。