arXiv:2608.23538stat.MEcs.LG2026-08

让复杂模型变得可解释,每条预测都生成一个透明的线性解释。

Interpretable AI with Local Distillation

论文配图:Interpretable AI with Local Distillation
图 1 · 摘自论文原文
  • 用局部线性模型逼近黑箱模型,通过加权相似预测点实现局部精准
  • 在17个数据集上接近黑箱模型精度,每个预测点输出稀疏线性解释
  • 能发现患者亚群差异,适合医疗等高风险需透明决策场景

现代人工智能模型如表格基础模型和梯度提升集成模型虽预测能力超越传统方法,但缺乏推理依据。高风险决策需要兼具准确与可解释性的模型。局部线性建模提供解决路径:光滑回归函数在局部可由线性近似,使每个查询点附近拟合线性模型即可保持高精度且透明。挑战在于确定‘局部’范围及构建解释工具。本文提出局部蒸馏:一个黑箱‘教师’指导每个查询点的正则化线性‘学生’模型。教师(1)通过加权预测结果相似的训练样本定义局部性,(2)以查询点的预测作为伪观测锚定拟合,其权重由数据估计。为实现解释性,对局部目标添加少量高斯随机扰动,通过多次重拟合评估稳定性:特征选择频率标识可靠特征,随机化拟合聚类识别稳定子群。在Lasso惩罚下,证明该随机化产生对训练响应微小扰动稳定的特征选择概率。在17个基准数据集上,局部蒸馏几乎匹配其AI教师的准确率,同时在每个测试点生成稀疏线性模型。在高维癌症基因表达案例中,框架识别出使用不同基因的患者亚群,这种异质性全局线性模型无法捕捉,黑箱模型也难以揭示。

原文摘要 · Abstract (English)

Modern AI models such as tabular foundation models and gradient-boosted ensembles can outpredict classical methods, but provide little basis for reasoning about their predictions. High-stakes decisions call for models that are both accurate and interpretable as built. Local linear modeling offers a path forward: a smooth regression function is locally well approximated by a linear one, allowing a linear fit near each query point to achieve high accuracy without sacrificing transparency. The challenges lie in learning what is "local" and developing statistical tools for interpretation. Here, we propose local distillation, in which a black-box "teacher" guides a regularized linear "student" model at each query point. The teacher (1) defines locality by upweighting training observations with similar predicted outcomes, and (2) anchors the fit with its prediction at the query point, included as a pseudo-observation whose weight is estimated from the data. For interpretation, we add a small amount of Gaussian randomization to the local objective and use refits to assess stability: selection frequencies identify reliable features at a query point, and clustering the randomized fits identifies stable subgroups across the data. Under the lasso penalty, we prove that this randomization yields feature-selection probabilities that are stable under small perturbations of the training responses. Across 17 benchmark datasets, local distillation nearly matches its AI teacher's accuracy while producing a sparse linear model at each test point. In a high-dimensional cancer gene expression example, the framework identifies patient subgroups whose local models use different genes; this heterogeneity is invisible to a global linear model, and difficult to surface in a black-box model.

可解释AI局部建模特征重要性医疗应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。