arXiv:2506.08928cs.LGstat.ME2025-06被引 3

提出局部特征重要性方法LMDI+,精准解释树模型在单个样本上的预测依据。

Local MDI+: Local Feature Importances for Tree-Based Models

  • 基于MDI+框架,结合树结构与线性回归,计算每个样本的局部特征重要性。
  • 在12个真实数据集上平均提升10%预测性能,且结果更稳定。
  • 适用于随机森林与梯度提升等模型,支持反事实分析与子群体发现。

树模型如随机森林因预测性能和计算效率,仍是表格数据的首选,尤其在高风险领域需可解释性保障。现有局部重要性方法如LIME和TreeSHAP依赖不稳定的扰动近似,忽略模型内部结构。全局版MDI+通过决策树与最小二乘在变换节点基下的等价性,融合树与线性重要性。但其全局分数无法解释个体差异。为此,本文提出局部MDI+(LMDI+),为每个样本量化特征重要性。在12个真实基准数据集上,LMDI+显著优于现有基线,在仅用选中特征时平均提升10%预测性能;且在不同随机种子下保持一致的重要性排序,稳定性更强。消融实验表明各组件均贡献显著,且优势扩展至梯度提升模型。最后,LMDI+成功识别分类任务中的近似反事实样本,并在住房数据案例中发现同质子群体。

原文摘要 · Abstract (English)

Tree-based ensembles such as random forests remain the go-to for tabular data over deep learning models due to their prediction performance and computational efficiency. These advantages have led to their widespread deployment in high-stakes domains, where interpretability is essential for ensuring trustworthy predictions. This has motivated the development of popular local feature importance methods such as LIME and TreeSHAP. However, these approaches rely on approximations that ignore the model's internal structure and instead depend on potentially unstable perturbations. These issues are addressed in the global setting by MDI+, a global feature importance method which combines tree-based and linear feature importances by exploiting an equivalence between decision trees and least squares on a transformed node basis. However, the global MDI+ scores are not able to explain predictions when faced with heterogeneous individual characteristics. To address this gap, we propose Local MDI+ (LMDI+), a novel extension of the MDI+ framework that quantifies feature importances for each particular sample. Across twelve real-world benchmark datasets, LMDI+ outperforms existing baselines at identifying instance-specific predictive features, yielding an average 10% improvement in predictive performance when using only the selected features. It further demonstrates greater stability by consistently producing similar instance-level feature importance rankings across repeated model fits with different random seeds. Ablation experiments show that each component of LMDI+ contributes to these gains, and that the improvements extend beyond random forests to gradient boosting models. Finally, we show that LMDI+ enables local interpretability use cases by identifying closely matched counterfactuals for each classification benchmark and discovering homogeneous subgroups in a housing dataset case study.

特征重要性树模型可解释性局部解释

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。