提出可信解释的双重标准,解决黑箱模型解释可靠性问题。
How can we trust opaque systems? Criteria for robust explanations in XAI
- 定义解释鲁棒性(ER)与方法鲁棒性(EMR)双重标准
- 指出单一方法鲁棒不足以保证解释可信
- 为可信AI解释提供可操作评估框架
深度学习算法在日常生活和科研中日益普及,但其预测准确的背后是极高的透明度缺失——无论是普通用户还是研究者都无法理解模型依赖数据的哪些特征进行决策。可信解释的关键标准是必须反映算法真实决策依据。尽管可解释人工智能(XAI)提供了有前景的解释方法,但近期对其性能的综述引发质疑。本文主张:可信解释的核心在于解释鲁棒性(ER),即不同XAI方法在相似情境下应产生一致解释。然而,若所有方法都给出相同但错误的解释,则仍不可信。因此,我们进一步提出每个XAI方法必须满足解释方法鲁棒性(EMR)这一先决条件。单独的方法鲁棒性不足以确保可信。本文构建并形式化了ER与EMR的评估准则,形成一套用于解释与建立对深度学习算法信任的框架,并探讨实际应用案例及未来方向。
原文摘要 · Abstract (English)
Deep learning (DL) algorithms are becoming ubiquitous in everyday life and in scientific research. However, the price we pay for their impressively accurate predictions is significant: their inner workings are notoriously opaque - it is unknown to laypeople and researchers alike what features of the data a DL system focuses on and how it ultimately succeeds in predicting correct outputs. A necessary criterion for trustworthy explanations is that they should reflect the relevant processes the algorithms' predictions are based on. The field of eXplainable Artificial Intelligence (XAI) presents promising methods to create such explanations. But recent reviews about their performance offer reasons for skepticism. As we will argue, a good criterion for trustworthiness is explanatory robustness: different XAI methods produce the same explanations in comparable contexts. However, in some instances, all methods may give the same, but still wrong, explanation. We therefore argue that in addition to explanatory robustness (ER), a prior requirement of explanation method robustness (EMR) has to be fulfilled by every XAI method. Conversely, the robustness of an individual method is in itself insufficient for trustworthiness. In what follows, we develop and formalize criteria for ER as well as EMR, providing a framework for explaining and establishing trust in DL algorithms. We also highlight interesting application cases and outline directions for future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。