arXiv:2512.23947cs.LGstat.ML2025-12NeurIPS被引 16

提出两种新损失函数,显著提升不平衡分类的准确率与理论保障。

Improved Balanced Classification with Theoretically Grounded Loss Functions

  • 设计GLA与GCA两类可理论保证的代理损失,解决类别不平衡问题。
  • GCA损失在极端不平衡下表现更优,其一致性边界优于GLA。
  • 实验验证新损失比传统加权法更有效,尤其适合小样本类别场景。

平衡损失广泛用于处理多分类中的类别不平衡问题,通过赋予所有类别相等重要性来防止少数类被忽略。然而,直接最小化该损失通常不可行,因此设计有效的代理损失成为关键。本文提出并研究两类先进代理损失:广义对数调整(GLA)损失和广义类别感知加权(GCA)损失。GLA将对数调整损失推广至更广泛的交叉熵家族,基于类别先验调整对数;GCA则在标准类别加权基础上引入类别依赖置信度阈值,并扩展到一般交叉熵框架。我们进行了全面的理论分析,发现GLA损失是贝叶斯一致的,但仅在完整假设集下是H一致的,其一致性界随最小类别概率倒数变化,至少为1/𝑝_min。而GCA损失在有界或完整假设集下均为H一致,其一致性界更优,为1/√𝑝_min,显著提升不平衡场景下的理论保障。实验表明,校准后的GCA损失与GLA损失均显著优于简单加权损失和对数调整损失。在常见基准上GLA略优,而在高度不平衡设置中GCA略胜。

原文摘要 · Abstract (English)

The balanced loss is a widely adopted objective for multi-class classification under class imbalance. By assigning equal importance to all classes, regardless of their frequency, it promotes fairness and ensures that minority classes are not overlooked. However, directly minimizing the balanced classification loss is typically intractable, which makes the design of effective surrogate losses a central question. This paper introduces and studies two advanced surrogate loss families: Generalized Logit-Adjusted (GLA) loss functions and Generalized Class-Aware weighted (GCA) losses. GLA losses generalize Logit-Adjusted losses, which shift logits based on class priors, to the broader general cross-entropy loss family. GCA loss functions extend the standard class-weighted losses, which scale losses inversely by class frequency, by incorporating class-dependent confidence margins and extending them to the general cross-entropy family. We present a comprehensive theoretical analysis of consistency for both loss families. We show that GLA losses are Bayes-consistent, but only $H$-consistent for complete (i.e., unbounded) hypothesis sets. Moreover, their $H$-consistency bounds depend inversely on the minimum class probability, scaling at least as $1/\mathsf p_{\min}$. In contrast, GCA losses are $H$-consistent for any hypothesis set that is bounded or complete, with $H$-consistency bounds that scale more favorably as $1/\sqrt{\mathsf p_{\min}}$, offering significantly stronger theoretical guarantees in imbalanced settings. We report the results of experiments demonstrating that, empirically, both the GCA losses with calibrated class-dependent confidence margins and GLA losses can greatly outperform straightforward class-weighted losses as well as the LA losses. GLA generally performs slightly better in common benchmarks, whereas GCA exhibits a slight edge in highly imbalanced settings.

分类不平衡损失函数理论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。