arXiv:2512.20006cs.LG2025-12

提出新型激活函数OGAB,缓解深度学习中的类别不平衡问题。

Orthogonal Activation with Implicit Group-Aware Bias Learning for Class Imbalance

  • 设计正交激活函数,增强少数类特征区分度。
  • 自动识别数据聚类并调整嵌入,提升分类可分性。
  • 无需标签信息,直接在训练中解决类别不平衡。

类别不平衡是机器学习与数据挖掘中的常见挑战,常导致分类器性能下降。尽管深度学习在特征提取方面表现优异,但在不平衡数据下性能仍会退化。本文提出一种新型激活函数OGAB,旨在缓解深度学习分类器中的类别不平衡问题。OGAB结合正交性与组感知偏差学习,可在不显式依赖标签信息的情况下,提升不平衡场景下的特征可区分性。核心洞察是:激活函数可通过引入强归纳偏置,解决超越传统非线性的复杂数据难题。正交变换能保持少数类信息,防止多数类主导嵌入空间;组感知偏差机制可自动识别数据聚类,并调整嵌入以增强类别可分性。与通过数据预处理或后处理纠正不平衡的方法不同,本方法在训练阶段的嵌入学习层面直接解决该问题,可无缝集成到学习过程中。我们在真实世界和合成的不平衡数据集上验证了该方法的有效性,结果表明其在性能上持续优于传统及可学习的激活函数。

原文摘要 · Abstract (English)

Class imbalance is a common challenge in machine learning and data mining, often leading to suboptimal performance in classifiers. While deep learning excels in feature extraction, its performance still deteriorates under imbalanced data. In this work, we propose a novel activation function, named OGAB, designed to alleviate class imbalance in deep learning classifiers. OGAB incorporates orthogonality and group-aware bias learning to enhance feature distinguishability in imbalanced scenarios without explicitly requiring label information. Our key insight is that activation functions can be used to introduce strong inductive biases that can address complex data challenges beyond traditional non-linearity. Our work demonstrates that orthogonal transformations can preserve information about minority classes by maintaining feature independence, thereby preventing the dominance of majority classes in the embedding space. Further, the proposed group-aware bias mechanism automatically identifies data clusters and adjusts embeddings to enhance class separability without the need for explicit supervision. Unlike existing approaches that address class imbalance through preprocessing data modifications or post-processing corrections, our proposed approach tackles class imbalance during the training phase at the embedding learning level, enabling direct integration with the learning process. We demonstrate the effectiveness of our solution on both real-world and synthetic imbalanced datasets, showing consistent performance improvements over both traditional and learnable activation functions.

类别不平衡激活函数嵌入学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。