提出新型二阶代理损失,实现在线AUC优化的高效与稳定
Online AUC Optimization Based on Second-order Surrogate Loss
- 用一阶二阶统计量替代成对损失,构建新代理函数
- 理论证明达到O(ln T)后悔界,优于传统O(√T)方法
- 适用于大规模、类别不平衡场景下的在线学习
AUC是分类任务中重要的性能指标,尤其在类别不平衡场景下。然而,由于成对0/1损失非凸且不连续,优化困难;同时实例级存储带来巨大内存开销,制约大规模应用。为此,本文提出基于成对铰链损失的新型二阶代理损失,并设计高效在线算法。不同于传统方法对每个成对损失项进行实例级近似,本方法直接用训练数据的一阶和二阶统计量构造整个聚合成对损失的代理函数。理论上,现有在线AUC优化算法通常达到O(√T)后悔界,而本文方法实现更紧致的O(ln T)边界。此外,通过核方法扩展至非线性场景。多个基准数据集上的实验表明,所提方法在在线AUC优化中兼具高效性与优越性。
原文摘要 · Abstract (English)
The Area Under the Curve (AUC) is an important performance metric for classification tasks, particularly in class-imbalanced scenarios. However, minimizing the AUC presents significant challenges due to the non-convex and discontinuous nature of pairwise 0/1 losses, which are difficult to optimize, as well as the substantial memory cost of instance-wise storage, which creates bottlenecks in large-scale applications. To overcome these challenges, we propose a novel second-order surrogate loss based on the pairwise hinge loss, and develop an efficient online algorithm. Unlike conventional approaches that approximate each individual pairwise 0/1 loss term with an instance-wise surrogate function, our approach introduces a new paradigm that directly substitutes the entire aggregated pairwise loss with a surrogate loss function constructed from the first- and second-order statistics of the training data. Theoretically, while existing online AUC optimization algorithms typically achieve an $\mathcal{O}(\sqrt{T})$ regret bound, our method attains a tighter $\mathcal{O}(\ln T)$ bound. Furthermore, we extend the proposed framework to nonlinear settings through a kernel-based formulation. Extensive experiments on multiple benchmark datasets demonstrate the superior efficiency and effectiveness of the proposed second-order surrogate loss in optimizing online AUC performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。