arXiv:2508.06064cs.AI2025-08

提出一种可随时输出高质量决策树的通用搜索算法,兼顾最优性和效率。

A Generic Complete Anytime Beam Search for Optimal Decision Tree

  • 基于重启式束搜索,逐步放宽剪枝条件以提升解质量。
  • 在标准数据集上,使用有限差异启发法的版本表现最佳,优于现有方法。
  • 框架可集成多种启发式策略,适合需要快速生成优质决策树的场景。

寻找最小化分类误差的最优决策树是已知的NP难问题。尽管基于MILP、CP、SAT或动态规划的精确算法能保证最优性,但常因搜索空间探索不平衡而表现出差的任意时行为——即在提前终止时难以快速获得高质量解。为此,已有若干精确方法的任意时扩展被提出,如LDS-DL8.5、Top-k-DL8.5和Blossom,但缺乏系统比较,难以评估其相对效果。本文提出CA-DL8.5,一种通用、完整且任意时的束搜索算法,扩展了DL8.5框架并统一了部分现有任意时策略。该算法通过模块化设计,可灵活集成多种启发式与松弛机制;复用DL8.5的高效分支限界剪枝与基于前缀树的缓存,并结合基于重启的束搜索,逐步放松剪枝条件以持续改进解的质量。实验基于标准分类基准,采用称为原始间隙积分的任意时评估指标,对比了基于纯度、增益、不一致性和Top-k启发式的多个实例。结果表明,采用有限差异(LDS)启发式的CA-DL8.5始终表现最优,优于其他变体及Blossom算法,同时保持完整性和最优性保证。

原文摘要 · Abstract (English)

Finding an optimal decision tree that minimizes classification error is known to be NP-hard. While exact algorithms based on MILP, CP, SAT, or dynamic programming guarantee optimality, they often suffer from poor anytime behavior -- meaning they struggle to find high-quality decision trees quickly when the search is stopped before completion -- due to unbalanced search space exploration. To address this, several anytime extensions of exact methods have been proposed, such as LDS-DL8.5, Top-k-DL8.5, and Blossom, but they have not been systematically compared, making it difficult to assess their relative effectiveness. In this paper, we propose CA-DL8.5, a generic, complete, and anytime beam search algorithm that extends the DL8.5 framework and unifies some existing anytime strategies. In particular, CA-DL8.5 generalizes previous approaches LDS-DL8.5 and Top-k-DL8.5, by allowing the integration of various heuristics and relaxation mechanisms through a modular design. The algorithm reuses DL8.5's efficient branch-and-bound pruning and trie-based caching, combined with a restart-based beam search that gradually relaxes pruning criteria to improve solution quality over time. Our contributions are twofold: (1) We introduce this new generic framework for exact and anytime decision tree learning, enabling the incorporation of diverse heuristics and search strategies; (2) We conduct a rigorous empirical comparison of several instantiations of CA-DL8.5 -- based on Purity, Gain, Discrepancy, and Top-k heuristics -- using an anytime evaluation metric called the primal gap integral. Experimental results on standard classification benchmarks show that CA-DL8.5 using LDS (limited discrepancy) consistently provides the best anytime performance, outperforming both other CA-DL8.5 variants and the Blossom algorithm while maintaining completeness and optimality guarantees.

决策树任意时搜索优化机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。