快速统一计算决策树模型的特征影响与交互,效率远超现有方法。
Woodelf++: A Fast and Unified Partial Dependence Plot Algorithm for Decision Tree Ensembles

- 基于伪布尔函数设计统一算法,同时支持多种可解释性工具
- 在40万样本数据上,计算速度比主流工具快6倍,交互分析快百万倍
- 适合需要高效模型解释的工业级机器学习应用
部分依赖图(PDP)用于可视化单个特征变化对模型平均预测的影响,广泛应用于决策树集成等模型的可解释性分析。联合部分依赖图扩展了该方法以揭示特征间的协同效应,而部分依赖交互值(PDIV)则量化特征间相互作用。任意顺序PDIV任务需计算所有特征子集在数据集每行上的交互值。本文提出Woodelf++,一种统一高效的算法,可在决策树集成上计算所有这些可解释性工具,基于先前用于高效SHAP计算的Woodelf算法。通过在伪布尔函数上定义合适度量,Woodelf++能统一计算精确与近似PDP、联合PDP及任意顺序PDIV。相比现有技术,其复杂度显著降低,尤其在任意顺序PDIV任务中实现指数级加速。此外,我们引入并高效计算全范围PDP,利用模型分裂阈值准确捕捉模型在所有可能特征值下的行为。Woodelf++采用纯Python实现并支持GPU加速。在40万行数据集上,其计算速度比当前最优方法快6倍,比scikit-learn快五阶数量级;对于任意顺序PDIV,Woodelf++仅需5分钟即可完成全部计算,而现有方法估计需超过100万年。
原文摘要 · Abstract (English)
Partial Dependence Plots (PDPs) visualize how changes in a single feature affect the average model prediction. They are widely used in practice to interpret decision tree ensembles and other machine learning models. Joint-PDPs extend this idea to pairs of features, revealing their combined effect. Partial Dependence Interaction Values (PDIVs) measure feature interactions. The Any-Order-PDIVs task computes these interactions for every feature subset across all rows of the dataset. We introduce Woodelf++, a unified and efficient approach for computing all these useful explainability tools on decision tree ensembles, building on Woodelf, an algorithm for efficient SHAP computation. By deriving suitable metrics over pseudo-Boolean functions, Woodelf++ can compute PDPs (exact and approximate), Joint-PDPs, and Any-Order-PDIVs in a unified framework. Our method delivers substantial complexity improvements over the state of the art, including an exponential gain for Any-Order-PDIVs. Additionally, we introduce and efficiently compute Full PDPs, which leverage the model's split thresholds to faithfully capture its behavior across all possible feature values. Woodelf++ is implemented in pure Python and supports GPU acceleration. On a dataset with 400,000 rows, Woodelf++ computes PDP and Joint-PDP up to 6x faster than the state of the art and up to five orders of magnitude faster than scikit-learn. For Any-Order-PDIVs, the gap is even larger: Woodelf++ computes all interaction values in 5 minutes, while the state of the art is estimated to require over 1,000,000 years.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。