arXiv:2602.00179cs.LG2026-02被引 1

解释不可靠的预测比解释本身更重要,稳定解释的前提是预测可信。

How Understanding Forecast Uncertainty Resolves the Explainability Problem in Machine Learning Models

  • 先判断预测是否可靠,仅在不确定性低时才生成局部线性解释
  • 决策边界处预测不确定性高,此时解释不稳定属正常现象
  • 非线性模型在部分区域不可靠时,应转用简单模型如逻辑回归

在关键决策中,机器学习模型的可解释性是核心关切,常为监管要求。局部线性解释方法(如LIME、SHAP)常被批评为在决策边界附近不稳定。本文指出,此类担忧源于对问题本质的误解:决策边界处预测不确定性本就高,因此解释不稳定是合理结果。正确做法应调整分析顺序——首先判断是否存在可用的低不确定性预测。若有,则可进行局部线性近似解释,此时解释稳定性也相应较高;若无,则应采用更简单的整体模型(如传统逻辑回归)。此外,一些宣称全域可解释的方法(如ReLU网络或任意分段线性模型),其实质是幻觉式可解释性,因其在分段边界处预测不确定性过高,无法提供有效指导。解释一个不可靠的预测毫无意义。

原文摘要 · Abstract (English)

For applications of machine learning in critical decisions, explainability is a primary concern, and often a regulatory requirement. Local linear methods for generating explanations, such as LIME and SHAP, have been criticized for being unstable near decision boundaries. In this paper, we explain that such concerns reflect a misunderstanding of the problem. The forecast uncertainty is high at decision boundaries, so consequently, the explanatory instability is high. The correct approach is to change the sequence of events and questions being asked. Nonlinear models can be highly predictive in some regions while having little or no predictability in others. Therefore, the first question is whether a usable forecast exists. When there is a forecast with low enough uncertainty to be useful, an explanation can be sought via a local linear approximation. In such cases, the explanatory instability is correspondingly low. When no usable forecast exists, the decision must fall to a simpler overall model such as traditional logistic regression. Additionally, these results show that some methods that purport to be explainable everywhere, such as ReLU networks or any piecewise linear model, have only an illusory explainability, because the forecast uncertainty at the segment boundaries is too high to be useful. Explaining an unusable forecast is pointless.

可解释性预测不确定性模型可信度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。