提出在任意乘积分布下构建决策树的理论保证,突破了以往仅限均匀分布的限制。
Decision Tree Learning on Product Spaces
- 基于自顶向下贪心策略,在任意乘积分布下构造近似决策树。
- 对大小为s、最大深度D_opt的最优树,构造出ε-近似树,大小上界为exp(Δ_opt D_opt log(e/ε))。
- 无需预先知道最优树大小或深度,适合实际应用且理论覆盖范围更广。
决策树学习是理论计算机科学中的核心课题,因其实际重要性而长期受关注。构建决策树的经典方法是自顶向下的贪心启发式算法,它递归地选择最具影响力变量进行分割。尽管该方法在实践中表现良好,但其理论分析一直受限。Blanc等(ITCS, 2020)首次为该贪心方法提供了严格的理论保证,但仅适用于均匀分布。本文将分析扩展到更一般且更贴近实际的任意乘积分布情形。我们的主要结果表明:对于任意可由大小为s、最大深度D_opt、平均深度Δ_opt的最优决策树计算的函数f,贪心启发式算法能构造出一个ε-近似树,其大小最多为exp(Δ_opt D_opt log(e/ε))。在最优树为满二叉树的特殊情况下,该界优于Blanc等人的结果,且适用分布类更广。此外,我们提出一种完全参数无关的算法——无需事先知晓最优树的大小或深度——相较于Blanc等人的方法具有显著的实用性优势。
原文摘要 · Abstract (English)
Decision tree learning has long been a central topic in theoretical computer science, driven by its practical importance. A fundamental and widely used method for decision tree construction is the top-down greedy heuristic, which recursively splits on the most influential variable. Despite its empirical success, theoretical analysis of this heuristic has been limited. A recent breakthrough by Blanc et al. (ITCS, 2020) provided the first rigorous theoretical guarantees for the greedy approach, but only under the uniform distribution. We extend this analysis to the more general and practically relevant setting of arbitrary product distributions. Our main result shows that for any function $f$ computable by an optimal decision tree of size $s$, maximum depth $D_{\text{opt}}$, and average depth $Δ_{\text{opt}}$, the greedy heuristic constructs an $ε$-approximating tree whose size grows at most with $\exp(Δ_{\text{opt}} D_{\text{opt}} \log(e/ε))$. In the special case where the optimal tree is a full binary tree, this bound improves upon the bound of Blanc et al. and holds under a strictly broader class of distributions. Moreover, we present an algorithm based on the top-down greedy heuristic that is entirely parameter-free -- it requires no prior knowledge of the optimal tree's size or depth -- offering a practical advantage over Blanc et al.'s method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。