通过分布式超参数训练,提升不平衡分类的性能与适应性。
Training Over a Distribution of Hyperparameters for Enhanced Performance and Adaptability on Imbalanced Classification
- 在损失函数上采样超参数分布进行联合训练
- 在CIFAR和医学图像任务中均超越单值训练效果
- 训练后可灵活调整超参数,无需重新训练
尽管二分类问题已得到广泛研究,但在严重类别不平衡条件下训练可靠分类器仍是挑战。近期方法通过修改损失函数或优化策略缓解不平衡带来的负面影响。我们观察到,不同损失函数的超参数在不同召回率下表现更优。为此提出基于损失条件训练(LCT)的方法:在一个超参数分布上训练单一模型,而非固定单一值。实验表明,该方法不仅逼近多个模型的性能,还在CIFAR及真实医学图像任务(如黑色素瘤、糖尿病视网膜病变检测)中实际提升了整体性能。此外,采用LCT训练的模型更具效率:部分超参数调优可在训练后完成,无需从头再训。
原文摘要 · Abstract (English)
Although binary classification is a well-studied problem, training reliable classifiers under severe class imbalance remains a challenge. Recent techniques mitigate the ill effects of imbalance on training by modifying the loss functions or optimization methods. We observe that different hyperparameter values on these loss functions perform better at different recall values. We propose to exploit this fact by training one model over a distribution of hyperparameter values--instead of a single value--via Loss Conditional Training (LCT). Experiments show that training over a distribution of hyperparameters not only approximates the performance of several models but actually improves the overall performance of models on both CIFAR and real medical imaging applications, such as melanoma and diabetic retinopathy detection. Furthermore, training models with LCT is more efficient because some hyperparameter tuning can be conducted after training to meet individual needs without needing to retrain from scratch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。