提出可严格求解任意分裂规则最优决策树的数学框架,统一多种经典数据结构。
Proper decision trees: An axiomatic framework for solving optimal decision tree problems with arbitrary splitting rules
- 基于公理化体系定义'正则决策树',涵盖K-D树、二分空间划分等
- 证明正则树可唯一表征为K-排列,非正则树复杂度显著更高
- 构建通用动态规划算法,支持深度、叶节点大小等约束
本文提出一个公理化框架,用于分析决策树的算法性质。该框架通过结构性和祖先约束对决策树问题进行分类,并聚焦于一类具有高灵活性与高效性的特殊问题——正则决策树。其灵活性体现在该类问题包含二分空间划分树、K-D树以及机器学习中的决策树模型;有效性体现为仅正则决策树能被唯一刻画为K-排列,而典型非正则决策树对应的是标签更复杂的二元标记树。基于此形式化特征,本文发展出一种通用算法,可针对任意分裂规则与目标函数精确求解正则决策树的最优问题。我们构造性地推导出通用动态规划递推式。然而,我们发现记忆化在空间复杂度上通常不可行,因需同时存储数据集与子树。这一结果反驳了文献中关于数据集与子树记忆化存在权衡的说法。框架还支持树深度、叶节点大小等约束,并可通过剪枝技术加速。最后,我们将分析扩展至若干非正则决策树,包括二值特征决策树、二叉搜索树及矩阵链乘法问题中的树结构,展示如何通过适当修改或放弃某些公理来解决这些问题。
原文摘要 · Abstract (English)
We present an axiomatic framework for analyzing the algorithmic properties of decision trees. This framework supports the classification of decision tree problems through structural and ancestral constraints within a rigorous mathematical foundation. The central focus of this paper is a special class of decision tree problems-which we term proper decision trees-due to their versatility and effectiveness. In terms of versatility, this class subsumes several well-known data structures, including binary space partitioning trees, K-D trees, and machine learning decision tree models. Regarding effectiveness, we prove that only proper decision trees can be uniquely characterized as K-permutations, whereas typical non-proper decision trees correspond to binary-labeled decision trees with substantially greater complexity. Using this formal characterization, we develop a generic algorithmic approach for solving optimal decision tree problems over arbitrary splitting rules and objective functions for proper decision trees. We constructively derive a generic dynamic programming recursion for solving these problems exactly. However, we show that memoization is generally impractical in terms of space complexity, as both datasets and subtrees must be stored. This result contradicts claims in the literature that suggest a trade-off between memoizing datasets and subtrees. Our framework further accommodates constraints such as tree depth and leaf size, and can be accelerated using techniques such as thinning. Finally, we extend our analysis to several non-proper decision trees, including the commonly studied decision tree over binary feature data, the binary search tree, and the tree structure arising in the matrix chain multiplication problem. We demonstrate how these problems can be solved by appropriately modifying or discarding certain axioms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。