从树模型中提取可解释规则,兼顾精度与简洁性。
Extracting Interpretable Models from Tree Ensembles: Computational and Statistical Perspectives
- 提出新算法,可灵活控制提取规则数量与交互深度
- 在多个数据集上优于现有规则提取方法,性能接近最优基准
- 适合需要模型可解释性的金融、医疗等高风险领域
树集成模型虽预测准确且能捕捉复杂交互关系,但难以解释。本文提出一种新估计器,可从中提取紧凑的决策规则集。该方法能联合控制规则数量和每条规则的交互深度,提升准确性。我们设计了精确算法求解优化问题,并开发近似算法计算正则化路径(不同规模下的解序列)。理论分析表明,在大样本下,该方法的预测误差与一个理想情形下的最优线性组合规则相当。实验显示,其性能显著优于现有规则提取算法。
原文摘要 · Abstract (English)
Tree ensembles are non-parametric methods widely recognized for their accuracy and ability to capture complex interactions. While these models excel at prediction, they are difficult to interpret and may fail to uncover useful relationships in the data. We propose an estimator to extract compact sets of decision rules from tree ensembles. The extracted models are accurate and can be manually examined to reveal relationships between the predictors and the response. A key novelty of our estimator is the flexibility to jointly control the number of rules extracted and the interaction depth of each rule, which improves accuracy. We develop a tailored exact algorithm to efficiently solve optimization problems underlying our estimator and an approximate algorithm for computing regularization paths, sequences of solutions that correspond to varying model sizes. We also establish novel non-asymptotic prediction error bounds for our proposed approach, comparing it to an oracle that chooses the best data-dependent linear combination of the rules in the ensemble subject to the same complexity constraint as our estimator. The bounds illustrate that the large-sample predictive performance of our estimator is on par with that of the oracle. Through experiments, we demonstrate that our estimator outperforms existing algorithms for rule extraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。