arXiv:2502.14001stat.MLcs.AI2025-02被引 1

用输入扰动分析解释医疗AI决策,不依赖训练数据。

Towards a perturbation-based explanation for medical AI as differentiable programs

  • 通过计算模型对输入微小扰动的响应变化(雅可比矩阵)来解释AI决策。
  • 该方法在不需额外数据的情况下,能客观评估模型对输入的敏感度。
  • 适合需要可解释性医疗AI系统的设计与临床部署场景。

近年来,机器学习算法的进步使得医疗设备能够集成人工智能(AI)模型以提供诊断支持和日常自动化。在医学和医疗领域,对AI模型输出结果的充分且客观的可解释性有特殊需求。然而,由于模型复杂性,现有AI通常被视为黑箱,其计算过程往往不透明。尽管已有多种方法通过评估各特征在分类与预测中的重要性来解释模型行为,但这些方法可能受训练或测试数据集规模和采样方案的影响而产生偏差。为克服现有方法的局限性,本文探索一种独立于学习过程、无需额外数据的客观解释新途径。作为初步研究,本文考察了深度学习模型雅可比矩阵的数值可行性,该矩阵衡量模型对输入微小扰动的响应稳定性。该指标可从已训练的AI模型中针对特定目标输入计算得出,是迈向基于扰动解释的第一步,有助于临床医生理解并解释AI模型在实际应用中的响应机制。

原文摘要 · Abstract (English)

Recent advancement in machine learning algorithms reaches a point where medical devices can be equipped with artificial intelligence (AI) models for diagnostic support and routine automation in clinical settings. In medicine and healthcare, there is a particular demand for sufficient and objective explainability of the outcome generated by AI models. However, AI models are generally considered as black boxes due to their complexity, and the computational process leading to their response is often opaque. Although several methods have been proposed to explain the behavior of models by evaluating the importance of each feature in discrimination and prediction, they may suffer from biases and opacities arising from the scale and sampling protocol of the dataset used for training or testing. To overcome the shortcomings of existing methods, we explore an alternative approach to provide an objective explanation of AI models that can be defined independently of the learning process and does not require additional data. As a preliminary study for this direction of research, this work examines a numerical availability of the Jacobian matrix of deep learning models that measures how stably a model responses against small perturbations added to the input. The indicator, if available, are calculated from a trained AI model for a given target input. This is a first step towards a perturbation-based explanation, which will assist medical practitioners in understanding and interpreting the response of the AI model in its clinical application.

医疗AI可解释性扰动分析雅可比矩阵

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。