用数据驱动的聚类方法减少决策树搜索空间,提速训练同时保持精度。
DICS: Data-Informed Centroid Splitting for Decision Tree Classifiers

- 基于聚类生成数据导向的候选分割点,缩小搜索范围。
- 在多个数据集上训练速度显著提升,精度与全搜索相当。
- 适用于分类树、随机森林和梯度提升模型,适合大规模数据场景。
决策树模型因可解释性和良好表现被广泛应用,但其训练过程在大规模高维数据下计算成本高昂,主要源于节点分裂时对候选分割点的穷举搜索。为此,本文提出数据驱动的质心分割方法(DICS),通过聚类构建紧凑且信息丰富的候选分割点集合,引入类别感知结构,显著缩小分类任务中的分裂搜索空间,同时保持预测性能。理论分析表明,在给定假设下,DICS不会降低分类树的性能。该方法可嵌入分类树、随机森林和梯度提升模型中。大量实验显示,DICS在合成及基准数据集上均实现相当精度,同时大幅缩短训练时间,验证了将数据先验融入分裂选择对可扩展分类树学习的有效性。
原文摘要 · Abstract (English)
Decision tree-based models are widely used in machine learning due to their interpretability and strong empirical performance. However, training decision trees can be computationally expensive, particularly for large and high-dimensional datasets, largely due to the exhaustive search over candidate splits at each node. To improve computational efficiency, we propose Data-Informed Centroid Splitting (DICS), a clustering-based framework that constructs a compact and informative set of candidate splits using data-driven priors. By incorporating class-aware structure, DICS significantly reduces the split search space for classification tasks while preserving predictive performance. We further provide theoretical analysis showing that under the stated assumptions, DICS does not degrade the performance of classification trees compared to exhaustive split search. DICS can be incorporated into classification trees, random forests, and gradient-boosting models. Extensive experiments demonstrate that DICS achieves comparable accuracy while substantially reducing training time across synthetic and benchmark datasets, highlighting the benefit of integrating data-informed priors into split selection for scalable classification tree learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。