用少量训练数据证明模型预测的可靠性,让解释更可信。
Proofs as Explanations: Short Certificates for Reliable Predictions
- 用子集数据作为证明,确保多数分类器一致预测
- 理论证明最短解释长度与模型复杂度相关
- 适合追求可解释性的高可靠性场景
我们提出一种可解释AI模型:对预测 $h(x)=y$,解释为训练数据的一个子集 $S'$,使得所有在 $S'$ 上最多犯 $b$ 个错误的分类器 $h' otin H$ 均预测 $h'(x)=y$。该子集 $S'$ 即为预测正确的短证明,前提是真实函数 $h^\star$ 属于 $H$ 且 $S$ 中至多有 $b$ 个噪声点。例如,当 $b=0$ 且 $H$ 为 $bR^d$ 中线性分类器时,若 $x$ 在正样本凸包内,则 Carathéodory 定理保证存在 $d+1$ 个点构成解释。本文推广此概念至一般假设类 $H$ 与任意 $b\geq 0$,定义了鲁棒空心星数(robust hollow star number),精确刻画最小证书的最坏情况大小,并分析其在自然类中的表现。还研究分布依赖的证书大小界,引入证书系数 $\varepsilon_x$,并证明样本量与 $\varepsilon_x$、$b$ 及 VC 维 $d$ 的上下界匹配。
原文摘要 · Abstract (English)
We consider a model for explainable AI in which an explanation for a prediction $h(x)=y$ consists of a subset $S'$ of the training data (if it exists) such that all classifiers $h' \in H$ that make at most $b$ mistakes on $S'$ predict $h'(x)=y$. Such a set $S'$ serves as a proof that $x$ indeed has label $y$ under the assumption that (1) the target function $h^\star$ belongs to $H$, and (2) the set $S$ contains at most $b$ corrupted points. For example, if $b=0$ and $H$ is the family of linear classifiers in $\mathbb{R}^d$, and if $x$ lies inside the convex hull of the positive data points in $S$ (and hence every consistent linear classifier labels $x$ as positive), then Carathéodory's theorem states that $x$ lies inside the convex hull of $d+1$ of those points. So, a set $S'$ of size $d+1$ could be released as an explanation for a positive prediction, and would serve as a short proof of correctness of the prediction under the assumption of realizability. In this work, we consider this problem more generally, for general hypothesis classes $H$ and general values $b\geq 0$. We define the notion of the robust hollow star number of $H$ (which generalizes the standard hollow star number), and show that it precisely characterizes the worst-case size of the smallest certificate achievable, and analyze its size for natural classes. We also consider worst-case distributional bounds on certificate size, as well as distribution-dependent bounds that we show tightly control the sample size needed to get a certificate for any given test example. In particular, we define a notion of the certificate coefficient $\varepsilon_x$ of an example $x$ with respect to a data distribution $D$ and target function $h^\star$, and prove matching upper and lower bounds on sample size as a function of $\varepsilon_x$, $b$, and the VC dimension $d$ of $H$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。