提出梯度归一化方法，让少数类不被忽略。
Harmonized Gradient Descent for Class Imbalanced Data Stream Online Learning
- 通过平衡各类梯度范数，改进在线学习
- 无需缓冲区或额外参数，通用性强
- 适合处理动态数据流中的类别不平衡
现实世界数据常以序列形式收集，且类别分布偏斜，形成类别不平衡的数据流。现有方法多采用重采样或重加权策略，本文则从训练过程入手，聚焦梯度下降技术，提出谐波梯度下降（HGD）算法，旨在使不同类别的梯度范数趋于一致。通过均衡梯度范数，HGD缓解了少数类的欠拟合问题，实现平衡的在线学习。该方法实现简洁，无需数据缓冲、额外参数或先验知识，可适配任意基于梯度下降优化的学习模型。在若干温和假设下，理论分析表明HGD具有满意的次线性遗憾界。在多个不平衡数据流场景中，与常用在线不平衡学习方法相比，实验验证了HGD在效率和有效性上的优势。
原文摘要 · Abstract (English)
Many real-world data are sequentially collected over time and often exhibit skewed class distributions, resulting in imbalanced data streams. While existing approaches have explored several strategies, such as resampling and reweighting, for imbalanced data stream learning, our work distinguishes itself by addressing the imbalance problem through training modification, particularly focusing on gradient descent techniques. We introduce the harmonized gradient descent (HGD) algorithm, which aims to equalize the norms of gradients across different classes. By ensuring the gradient norm balance, HGD mitigates under-fitting for minor classes and achieves balanced online learning. Notably, HGD operates in a streamlined implementation process, requiring no data-buffer, extra parameters, or prior knowledge, making it applicable to any learning models utilizing gradient descent for optimization. Theoretical analysis, based on a few common and mild assumptions, shows that HGD achieves a satisfied sub-linear regret bound. The proposed algorithm are compared with the commonly used online imbalance learning methods under several imbalanced data stream scenarios. Extensive experimental evaluations demonstrate the efficiency and effectiveness of HGD in learning imbalanced data streams.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。