用统计检验替代传统分裂方式,让决策树更科学、更易调参。
ZTree: A Subgroup Identification Based Decision Tree Learning Framework
- 以假设检验识别显著子群体,取代传统纯度判据
- 在5个大规模数据集上表现优异,低数据量下优势明显
- 仅需调节一个阈值,自动控制复杂度,适合科研与工程落地
决策树因可解释性强被广泛使用,但传统方法依赖纯度指标进行分裂。本文提出ZTree框架,采用统计检验(如z检验、t检验、Mann-Whitney U、log-rank)判断候选子群体与补集是否存在显著差异,实现基于统计学原理的分裂。通过交叉验证修正多重检验问题,动态决定是否继续分裂,从而无需后剪枝。树复杂度仅由一个z阈值控制,该阈值等价于p值,使参数调整直观高效。一旦以最宽松阈值训练出完整树,所有简化版本可通过移除不满足更高阈值的节点获得。在5个大型UCI数据集上的实验表明,ZTree在低数据场景下表现突出,且生成的树更简洁,性能不降。该方法为决策树分裂提供了统计严谨的新范式。
原文摘要 · Abstract (English)
Decision trees are a commonly used class of machine learning models valued for their interpretability and versatility, capable of both classification and regression. We propose ZTree, a novel decision tree learning framework that replaces CART's traditional purity based splitting with statistically principled subgroup identification. At each node, ZTree applies hypothesis testing (e.g., z-tests, t-tests, Mann-Whitney U, log-rank) to assess whether a candidate subgroup differs meaningfully from the complement. To adjust for the complication of multiple testing, we employ a cross-validation-based approach to determine if further node splitting is needed. This robust stopping criterion eliminates the need for post-pruning and makes the test threshold (z-threshold) the only parameter for controlling tree complexity. Because of the simplicity of the tree growing procedure, once a detailed tree is learned using the most lenient z-threshold, all simpler trees can be derived by simply removing nodes that do not meet the larger z-thresholds. This makes parameter tuning intuitive and efficient. Furthermore, this z-threshold is essentially a p-value, allowing users to easily plug in appropriate statistical tests into our framework without adjusting the range of parameter search. Empirical evaluation on five large-scale UCI datasets demonstrates that ZTree consistently delivers strong performance, especially at low data regimes. Compared to CART, ZTree also tends to grow simpler trees without sacrificing performance. ZTree introduces a statistically grounded alternative to traditional decision tree splitting by leveraging hypothesis testing and a cross-validation approach to multiple testing correction, resulting in an efficient and flexible framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。