提出四类最优决策树的严格定义与通用算法,统一解决任意分割规则下的优化问题。
Foundational theory for optimal decision tree problems. I. Algorithmic and geometric foundations
- 基于代数编程理论,用可执行递归程序明确定义四类最优决策树问题。
- 推导出四种新算法,涵盖已知轴平行决策树为特例,且支持任意分割规则。
- 框架可扩展至混合分割规则,适合研究高效、灵活的决策树方法者参考。
在本系列论文的第一部分(第I篇)中,我们提出了四类新的最优决策树(ODT)问题定义:三类针对大小约束的树,一类针对深度约束的树。这些定义通过可执行的递归程序明确表述,满足我们提出的全部形式化标准,类似于通用求解器中的“标准形式”。基于代数编程理论——一种从规格推导正确性保证算法的关系形式化方法,我们不仅能判定动态规划解的存在性或不存在性,还能在存在时构造性地推导出解。因此,这四个通用问题定义导出了四种针对任意满足公理和目标函数形式的分割规则的最优算法。这些算法涵盖了已知的深度约束轴平行决策树算法作为特例,同时为一般最优决策树问题提供了统一、高效且优雅的解决方案。在第二部分中,我们提出了首个最优超曲面决策树算法,并与轴平行决策树算法(包括启发式CART和最先进的最优方法)进行了全面实验。结果表明,具有灵活分割规则的决策树具有显著潜力。此外,我们的框架可轻松扩展,以支持构建更灵活的决策树算法,包括混合分割规则的情况。
原文摘要 · Abstract (English)
In the first paper (part I) of this series of two, we introduce four novel definitions of the ODT problems: three for size-constrained trees and one for depth-constrained trees. These definitions are stated unambiguously through executable recursive programs, satisfying all criteria we propose for a formal specification. In this sense, they resemble the "standard form" used in the study of general-purpose solvers. Grounded in algebraic programming theory-a relational formalism for deriving correct-by-construction algorithms from specifications-we can not only establish the existence or nonexistence of dynamic programming solutions but also derive them constructively whenever they exist. Consequently, the four generic problem definitions yield four novel optimal algorithms for ODT problems with arbitrary splitting rules that satisfy the axioms and objective functions of a given form. These algorithms encompass the known depth-constrained, axis-parallel ODT algorithm as the special case, while providing a unified, efficient, and elegant solution for the general ODT problem. In Part II, we present the first optimal hypersurface decision tree algorithm and provide comprehensive experiments against axis-parallel decision tree algorithms, including heuristic CART and state-of-the-art optimal methods. The results demonstrate the significant potential of decision trees with flexible splitting rules. Moreover, our framework is readily extendable to support algorithms for constructing even more flexible decision trees, including those with mixed splitting rules.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。