arXiv:2504.04371cs.LGstat.ML2025-04被引 1

改进SVM在非欧空间的分类效果,通过协方差调整提升精度。

An Algorithm to perform Covariance-Adjusted Support Vector Classification in Non-Euclidean Spaces

  • 引入协方差结构,用Cholesky分解调整非欧空间中的SVM优化问题。
  • 在多个数据集上,准确率、F1分数和ROC性能显著优于传统SVM。
  • 适合处理高维非欧数据,尤其在类内协方差复杂的场景中表现优异。

传统支持向量机(SVM)分类依赖于最大间隔原则,在欧氏空间中通过KKT边界条件求解最优分类器。本文指出,该原则仅在欧氏空间中成立,而在非欧空间中,最大间隔本质依赖于类内数据协方差,导致标准KKT建模次优。为此,本文提出一种新算法:基于类协方差矩阵的Cholesky分解,将协方差信息嵌入优化目标,构建协方差调整型SVM(Cholesky-SVM)。该方法通过迭代估计样本协方差以逼近总体协方差,实现非欧空间中的最优分类。实验表明,该模型在多个数据集上的准确率、精确率、F1得分及ROC性能均显著优于线性与核SVM。结果验证了其在非欧空间中的有效性与鲁棒性。

原文摘要 · Abstract (English)

Traditional Support Vector Machine (SVM) classification is carried out by finding the max-margin classifier for the training data that divides the margin space into two equal sub-spaces. This study demonstrates limitations of performing Support Vector Classification in non-Euclidean spaces by establishing that the underlying principle of max-margin classification and Karush Kuhn Tucker (KKT) boundary conditions are optimal only in the Euclidean vector spaces. The study establishes a methodology to perform Support Vector Classification in Non-Euclidean Spaces by incorporating data covariance into the optimization problem using Cholesky Decomposition of respective class covariance structure. It also demonstrates that in non-Euclidean spaces KKT modelling is sub-optimal as the principle of maximum margin is a function of intra-class data covariances and the classifier obtained separates the margin space in ratio of the respective class population covariance matrix. The study proposes an algorithm to iteratively estimate the population covariance-adjusted SVM classifier in non-Euclidean space from sample covariance matrices of the training data. The effectiveness of this SVM classification approach is demonstrated by applying the classifier on multiple datasets and comparing the performance with traditional SVM kernels and whitening algorithms. The Cholesky-SVM model shows marked improvement in the accuracy, precision, F1 scores and ROC performance compared to linear and other kernel SVMs.

SVM非欧空间协方差调整分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。