针对标签不确定的不平衡数据,提出更鲁棒的多分类SVM模型
Wasserstein Distributionally Robust Multiclass Support Vector Machine
- 基于Wasserstein分布鲁棒优化,改进Crammer-Singer损失函数
- 在不平衡数据上显著优于当前最优OVA模型,提升分类稳定性
- 适合处理标签噪声大、类别分布不均的真实场景
我们研究数据特征 $\mathbf{x}$ 与标签 $\mathbf{y}$ 均存在不确定性的多分类问题。发现传统分布鲁棒一对多(OVA)分类器在数据不平衡时表现不佳。为此,利用Wasserstein分布鲁棒优化,构建基于Crammer-Singer(CS)损失的鲁棒多分类支持向量机。首先证明CS损失对所有 $\mathbf{x} \in \mathcal{X}$ 与 $\mathbf{y} \in \mathcal{Y}$ 均有上界且为Lipschitz连续;借助强对偶性,将最坏情况风险问题转化为可解凸形式,得益于CS损失的正则性。进一步提出核版本模型以处理非线性分类边界,并给出可计算的凸上界。针对线性情形设计投影子梯度算法以提升可扩展性。数值实验表明,本模型在高度不平衡训练数据下优于现有最优OVA模型;在多个真实世界数据集上,相比正则化模型,其因显式建模标签不确定性而表现更优。
原文摘要 · Abstract (English)
We study the problem of multiclass classification for settings where data features $\mathbf{x}$ and their labels $\mathbf{y}$ are uncertain. We identify that distributionally robust one-vs-all (OVA) classifiers often struggle in settings with imbalanced data. To address this issue, we use Wasserstein distributionally robust optimization to develop a robust version of the multiclass support vector machine (SVM) characterized by the Crammer-Singer (CS) loss. First, we prove that the CS loss is bounded from above by a Lipschitz continuous function for all $\mathbf{x} \in \mathcal{X}$ and $\mathbf{y} \in \mathcal{Y}$, then we exploit strong duality results to express the dual of the worst-case risk problem, and we show that the worst-case risk minimization problem admits a tractable convex reformulation due to the regularity of the CS loss. Moreover, we develop a kernel version of our proposed model to account for nonlinear class separation, and we show that it admits a tractable convex upper bound. We also propose a projected subgradient method algorithm for a special case of our proposed linear model to improve scalability. Our numerical experiments demonstrate that our model outperforms state-of-the art OVA models in settings where the training data is highly imbalanced. We also show through experiments on popular real-world datasets that our proposed model often outperforms its regularized counterpart as the first accounts for uncertain labels unlike the latter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。