arXiv:2604.25158stat.MLcs.LG2026-04

让SVM学习权威模型的分类偏好,提升预测准确性。

Elite-Driven Support Vector Machines for Classification

论文配图:Elite-Driven Support Vector Machines for Classification
图 1 · 摘自论文原文
  • 通过精英样本引导松弛变量,融合参考模型的分类偏好。
  • 在多个UCI数据集上表现优于或媲美经典SVM方法。
  • 无需额外特征,适合希望继承专家模型决策的场景。

支持向量机(SVM)是二分类的标准工具,但其传统形式完全依赖数据,无法直接融入可信基准模型或对特定数据子集的结构化偏好。本文提出精英驱动支持向量机(EDSVM),通过在正则化经验风险最小化中引入对精选精英样本松弛变量的引导,使其偏离参考模型的松弛值时施加惩罚,从而定义出局部、边缘对齐的参考模型接近度。该框架下构建了基于合页损失和平方松弛损失的两种模型(C-EDSVM与LS-EDSVM),均能通过标准SVM求解器的微小修改实现,并给出分类校准的充分条件。模拟实验及多个UCI基准测试表明,EDSVM能紧密跟踪参考SVM的行为,且预测性能可媲美甚至超越C-SVM、LINEX-SVM与LS-SVM。

原文摘要 · Abstract (English)

Support vector machines (SVMs) are a standard tool for binary classification, but their classical formulations are purely data-driven and offer no direct way to encode trusted benchmark models or structured preferences on selected subsets of the data. We propose Elite-Driven Support Vector Machines (EDSVM), a general framework that augments regularized empirical risk minimization by guiding the slack variables for a curated set of elite observations (typically the union of support vectors from one or more reference SVMs). EDSVM combines the usual slack loss with a deviation penalty that shrinks new slacks toward benchmark slack values, defining a localized, margin-aligned notion of proximity to reference models, unlike global function penalties in knowledge distillation or teacher-student methods, and without requiring privileged features as in SVM+/LUPI. Within this framework we develop two concrete models, C-EDSVM and LS-EDSVM, based respectively on hinge-type and squared-slack losses. For both variants we derive dual quadratic programs that can be implemented with modest modifications of standard SVM solvers, and we give simple sufficient conditions under which the induced margin losses are classification calibrated. Simulation studies and experiments on several UCI benchmarks show that EDSVMs closely track the behaviour induced by reference SVMs while achieving predictive performance that is competitive with, and sometimes better than, C-SVM, LINEX-SVM, and LS-SVM.

SVM模型集成分类优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。