发现决策树在特定条件下反而偏爱少数类,挑战了传统认知。
Analyzing decision tree bias towards the minority class
- 通过分析预测变量条件下的结果分布,揭示偏差根源
- 纯度最优的决策树在特定条件下偏向少数类
- 可通过剪枝、深度限制或后校准缓解该偏差
长期以来普遍认为机器学习模型在处理类别不平衡数据时会偏向多数类,从而忽略少数类。然而,近期一项模拟研究发现决策树可能反而偏向少数类,引发矛盾。本文批判性回顾相关文献,指出以往研究忽视了给定预测变量下结果的条件分布,导致错误结论。我们证明,在特定条件下,以纯度为目标拟合的决策树确实会偏向少数类,推翻了决策树始终偏向多数类的普遍看法。该偏差可通过剪枝、设置最大树深等正则化方法,或使用后处理校准技术减轻。研究结果对随机森林等主流树模型的应用具有重要启示:尽管随机森林由纯度最优的决策树构成,但这一做法可能并非最优选择。
原文摘要 · Abstract (English)
There is a widespread and longstanding belief that machine learning models are biased towards the majority class when learning from imbalanced binary response data, leading them to neglect or ignore the minority class. Motivated by a recent simulation study that found that decision trees can be biased towards the minority class, our paper aims to reconcile the conflict between that study and other published works. First, we critically evaluate past literature on this problem, finding that failing to consider the conditional distribution of the outcome given the predictors has led to incorrect conclusions about the bias in decision trees. We then show that, under specific conditions, decision trees fit to purity are biased towards the minority class, debunking the belief that decision trees are always biased towards the majority class. This bias can be reduced by adjusting the tree-fitting process to include regularization methods like pruning and setting a maximum tree depth, and/or by using post-hoc calibration methods. Our findings have implications on the use of popular tree-based models, such as random forests. Although random forests are often composed of decision trees fit to purity, our work adds to recent literature indicating that this may not be the best approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。