提出FastPD方法,高效准确估计任意特征子集的局部依赖函数。
Fast Estimation of Partial Dependence Functions using Trees
- 基于树结构设计新算法,实现对局部依赖函数的快速估计。
- 在特征相关时保持一致性,且计算复杂度从二次降至线性。
- 可统一生成SHAP、PD图及高阶交互效应,适合模型解释需求者。
现有解释方法多基于局部依赖(PD)函数,用于刻画特定特征子集对模型预测的影响,通过平均其余特征实现。代表性方法包括基于博弈论的SHAP值和一维PD图,后者反映平均边际主效应。近期研究指出SHAP值可能混淆主效应与交互效应。尽管如此,相较于其他基于PD的方法,SHAP的优势在于存在快速估算技术如TreeSHAP。本文提出一种新的基于树的估计器FastPD,可高效估计任意特征子集的PD函数。我们证明FastPD能一致地估计目标总体量,而路径依赖的TreeSHAP在特征相关时具有不一致性。对于中等深度的树,FastPD将计算复杂度从二次降低至线性。通过估计任意特征子集的PD函数,FastPD可用于提取包括SHAP、PD图及高阶交互效应在内的多种基于PD的解释。
原文摘要 · Abstract (English)
Many existing interpretation methods are based on Partial Dependence (PD) functions that, for a pre-trained machine learning model, capture how a subset of the features affects the predictions by averaging over the remaining features. Notable methods include Shapley additive explanations (SHAP) which computes feature contributions based on a game theoretical interpretation and PD plots (i.e., 1-dim PD functions) that capture average marginal main effects. Recent work has connected these approaches using a functional decomposition and argues that SHAP values can be misleading since they merge main and interaction effects into a single local effect. However, a major advantage of SHAP compared to other PD-based interpretations has been the availability of fast estimation techniques, such as \texttt{TreeSHAP}. In this paper, we propose a new tree-based estimator, \texttt{FastPD}, which efficiently estimates arbitrary PD functions. We show that \texttt{FastPD} consistently estimates the desired population quantity -- in contrast to path-dependent \texttt{TreeSHAP} which is inconsistent when features are correlated. For moderately deep trees, \texttt{FastPD} improves the complexity of existing methods from quadratic to linear in the number of observations. By estimating PD functions for arbitrary feature subsets, \texttt{FastPD} can be used to extract PD-based interpretations such as SHAP, PD plots and higher-order interaction effects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。