将分类不平衡视为迁移学习,揭示了数据增强策略的优劣原理。
Classification Imbalance as Transfer Learning
- 把少数类数据生成看作从源分布到目标分布的迁移学习。
- 在高维场景下,随机重采样比SMOTE的迁移代价更低,性能更优。
- 为不平衡分类的数据增强方法选择提供了理论依据。
当某一类别远少于另一类别时,便出现分类不平衡问题。本文将此情形建模为标签(先验)偏移下的迁移学习:观测数据产生的不平衡源分布与评估性能所用的平衡目标分布之间存在差异。在此框架下,研究了一类通过从估计的少数类分布中生成合成样本以实现类别平衡的过采样方法,其中著名的SMOTE算法是典型代表。我们证明,额外风险可分解为在平衡训练下可达到的速率与一个附加项——迁移成本,后者衡量了估计的少数类分布与真实分布之间的差异。特别地,我们发现,在中等高维情况下,SMOTE的迁移成本显著高于随机重采样(bootstrapping),表明后者通常表现更优。实验结果验证了这些发现。总体而言,本研究为不平衡分类中的数据增强策略选择提供了指导。
原文摘要 · Abstract (English)
Classification imbalance arises when one class is much rarer than the other. We frame this setting as transfer learning under label (prior) shift between an imbalanced source distribution induced by the observed data and a balanced target distribution under which performance is evaluated. Within this framework, we study a family of oversampling procedures that augment the training data by generating synthetic samples from an estimated minority-class distribution to roughly balance the classes, among which the celebrated SMOTE algorithm is a canonical example. We show that the excess risk decomposes into the rate achievable under balanced training (as if the data had been drawn from the balanced target distribution) and an additional term, the cost of transfer, which quantifies the discrepancy between the estimated and true minority-class distributions. In particular, we show that the cost of transfer for SMOTE dominates that of bootstrapping (random oversampling) in moderately high dimensions, suggesting that we should expect bootstrapping to have better performance than SMOTE in general. We corroborate these findings with experimental evidence. More broadly, our results provide guidance for choosing among augmentation strategies for imbalanced classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。