修正模型学习到的先验分布,提升长尾识别中少数类性能
Prior2Posterior: Model Prior Correction for Long-Tailed Learning
- 训练后通过后处理调整后验概率,校正因数据不平衡带来的先验偏差
- 在多个长尾数据集上实现新最优效果,尤其提升尾部类别准确率
- 无需重新训练即可改进现有方法,适用于各类长尾学习模型
基于学习的长尾识别方法在平衡测试集上泛化能力不足,因其学习到的后验分布受高频类别(头类)主导,导致低频类别(尾类)表现差。我们发现训练结束时模型实际学习到的有效先验可能与基于样本频率的原始先验不同。为此,提出一种新方法:利用后验概率准确建模训练后模型的有效先验,并在训练后以后处理方式校正预测的后验概率(称为Prior2Posterior, P2P)。理论分析表明该方法对朴素交叉熵损失和对数调整损失均具最优性。实验显示,该方法在多个长尾基准数据集上达到当前最优性能,且可无缝应用于已有方法,无需重训练即可进一步提升其表现。
原文摘要 · Abstract (English)
Learning-based solutions for long-tailed recognition face difficulties in generalizing on balanced test datasets. Due to imbalanced data prior, the learned \textit{a posteriori} distribution is biased toward the most frequent (head) classes, leading to an inferior performance on the least frequent (tail) classes. In general, the performance can be improved by removing such a bias by eliminating the effect of imbalanced prior modeled using the number of class samples (frequencies). We first observe that the \textit{effective prior} on the classes, learned by the model at the end of the training, can differ from the empirical prior obtained using class frequencies. Thus, we propose a novel approach to accurately model the effective prior of a trained model using \textit{a posteriori} probabilities. We propose to correct the imbalanced prior by adjusting the predicted \textit{a posteriori} probabilities (Prior2Posterior: P2P) using the calculated prior in a post-hoc manner after the training, and show that it can result in improved model performance. We present theoretical analysis showing the optimality of our approach for models trained with naive cross-entropy loss as well as logit adjusted loss. Our experiments show that the proposed approach achieves new state-of-the-art (SOTA) on several benchmark datasets from the long-tail literature in the category of logit adjustment methods. Further, the proposed approach can be used to inspect any existing method to capture the \textit{effective prior} and remove any residual bias to improve its performance, post-hoc, without model retraining. We also show that by using the proposed post-hoc approach, the performance of many existing methods can be improved further.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。