arXiv:2605.07837cs.LGcs.AI2026-05中稿 · JMLR

无需近似即可训练精准的可解释决策树,提升回归与分类性能。

Approximation-Free Differentiable Oblique Decision Trees

  • 用可逆神经网络表示硬性斜向决策树,实现端到端梯度训练。
  • 在分类和回归任务上均超越现有最先进方法,尤其在回归中表现突出。
  • 适用于强化学习中的程序化策略,拓展了决策树的应用场景。

决策树因可解释性好,在医疗诊断等安全关键领域广泛应用,尤其擅长处理表格数据。然而,训练高精度的斜向决策树面临优化复杂、过拟合风险,尤其在回归任务中更为显著。近年研究提出可微分框架,支持梯度训练并联合优化划分边界与叶节点回归器,但大多依赖近似方法,如通过概率软化边界(软决策树)或使用量化梯度(如直通估计器STE)。为此,本文提出DTSemNet,一种语义等价且可逆的硬性斜向决策树神经网络表示,支持标准梯度下降的端到端训练,无需任何近似。尽管分类任务天然适配该框架,回归任务仍因内部节点与叶节点回归器联合优化而具挑战。为此,我们分析了STE的局限性,并引入一种退火式Top-k方法,提供精确梯度信号而不依赖近似。大量实验表明,基于DTSemNet训练的斜向决策树在分类与回归基准上均优于当前最优可微分决策树。此外,我们证明其可作为强化学习环境中的程序化决策树策略,进一步拓展应用范围。

原文摘要 · Abstract (English)

Decision Trees (DTs) are widely used in safety-critical domains such as medical diagnosis, valued for their interpretability and effectiveness on tabular data. However, training accurate oblique DTs is challenging due to complex optimization landscapes and overfitting risks, particularly in regression. Recent advances have introduced differentiable formulations that enable gradient-based training and joint optimization of decision boundaries and leaf regressors. Yet, existing approaches typically rely on approximations, either through probabilistic softening of boundaries (soft DTs) or quantized gradients such as the Straight-Through Estimator (STE). To overcome these limitations, we propose DTSemNet, a novel, semantically equivalent, and invertible representation of hard oblique DTs as neural networks. DTSemNet enables end-to-end training with standard gradient descent, eliminating the need for approximations in both classification and regression. While classification aligns naturally with this formulation, regression remains challenging due to the joint optimization of internal nodes and leaf regressors. To address this, we analyze the limitations of STE and introduce an annealed Top-k method that provides accurate gradient signals without approximation. Extensive experiments on classification and regression benchmarks show that DTSemNet-trained oblique DTs outperform state-of-the-art differentiable DTs. Furthermore, we demonstrate that DTSemNet can serve as programmatic DT policies in reinforcement learning environments, thereby broadening their applicability.

决策树可微分回归强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。