arXiv:2410.11449cs.LG2024-10NeurIPS被引 5

用直方图树建模条件分布,兼顾准确与可解释性。

Conditional Density Estimation with Histogram Trees

  • 以最小描述长度原则构建直方图决策树,无需调参正则化
  • 相比现有方法,对无关特征更鲁棒,且在对数损失上更优
  • 树结构更小,适合需要透明决策过程的场景

条件密度估计(CDE)不仅预测条件均值,还建模完整条件分布,提供比回归更丰富的数据理解,尤其适用于关键领域。然而,可解释的CDE方法研究不足。现有方法多采用基于核的方法,而概念简单、可视化友好的树结构方法在CDE中被忽视。为此,我们提出条件密度树(CDTree),一种全非参数模型:决策树每个叶节点由直方图模型构成。我们基于最小描述长度(MDL)原则形式化学习问题,避免正则化超参数调优。进一步设计一种迭代算法,虽为贪心策略,但搜索每种可能节点分割下的最优直方图。实验表明,相较于现有可解释的CDE方法,CDTree在对数损失上更准确,对无关特征更鲁棒,且树规模更小,提升可解释性。

原文摘要 · Abstract (English)

Conditional density estimation (CDE) goes beyond regression by modeling the full conditional distribution, providing a richer understanding of the data than just the conditional mean in regression. This makes CDE particularly useful in critical application domains. However, interpretable CDE methods are understudied. Current methods typically employ kernel-based approaches, using kernel functions directly for kernel density estimation or as basis functions in linear models. In contrast, despite their conceptual simplicity and visualization suitability, tree-based methods -- which are arguably more comprehensible -- have been largely overlooked for CDE tasks. Thus, we propose the Conditional Density Tree (CDTree), a fully non-parametric model consisting of a decision tree in which each leaf is formed by a histogram model. Specifically, we formalize the problem of learning a CDTree using the minimum description length (MDL) principle, which eliminates the need for tuning the hyperparameter for regularization. Next, we propose an iterative algorithm that, although greedily, searches the optimal histogram for every possible node split. Our experiments demonstrate that, in comparison to existing interpretable CDE methods, CDTrees are both more accurate (as measured by the log-loss) and more robust against irrelevant features. Further, our approach leads to smaller tree sizes than existing tree-based models, which benefits interpretability.

条件密度估计可解释模型决策树直方图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。