揭示主流可解释AI在脑影像中的系统性失效,提出安全应用新方案
Explainable AI Methods for Neuroimaging: Systematic Failures of Common Tools, the Need for Domain-Specific Validation, and a Proposal for Safe Application
- 构建新验证框架,用已知信号源测试4.5万例脑部MRI的解释效果
- GradCAM与LRP方法均严重误判,仅SmoothGrad保持稳定准确
- 呼吁神经影像领域定制XAI工具,避免错误解读研究结论
可信的深度学习模型解释对神经影像应用至关重要,但常用可解释AI(XAI)方法缺乏严格验证,存在误读风险。我们首次在约4.5万例结构化脑部MRI上,通过新型XAI验证框架系统比较多种方法。该框架通过设定具有已知信号来源的预测任务(如局部解剖特征或受试者特异性病灶)建立可验证的真值,无需人为修改图像。分析显示,两种最广泛使用的方法存在系统性失败:GradCAM始终无法准确定位预测特征,而层间显著性传播(LRP)生成大量人工伪影,表明其不适应神经影像数据特性。结果表明,这些失败源于领域错配——面向自然图像设计的方法需大幅调整才能用于神经影像。相比之下,假设更少、基于梯度的SmoothGrad方法始终表现准确,说明其概念简洁性使其对领域偏移更具鲁棒性。研究强调必须为神经影像领域定制和验证XAI方法,提示此前使用标准XAI的研究结论需重新审视,并为实际应用提供紧迫指导。
原文摘要 · Abstract (English)
Trustworthy interpretation of deep learning models is critical for neuroimaging applications, yet commonly used Explainable AI (XAI) methods lack rigorous validation, risking misinterpretation. We performed the first large-scale, systematic comparison of XAI methods on ~45,000 structural brain MRIs using a novel XAI validation framework. This framework establishes verifiable ground truth by constructing prediction tasks with known signal sources - from localized anatomical features to subject-specific clinical lesions - without artificially altering input images. Our analysis reveals systematic failures in two of the most widely used methods: GradCAM consistently failed to localize predictive features, while Layer-wise Relevance Propagation generated extensive, artifactual explanations that suggest incompatibility with neuroimaging data characteristics. Our results indicate that these failures stem from a domain mismatch, where methods with design principles tailored to natural images require substantial adaptation for neuroimaging data. In contrast, the simpler, gradient-based method SmoothGrad, which makes fewer assumptions about data structure, proved consistently accurate, suggesting its conceptual simplicity makes it more robust to this domain shift. These findings highlight the need for domain-specific adaptation and validation of XAI methods, suggest that interpretations from prior neuroimaging studies using standard XAI methodology warrant re-evaluation, and provide urgent guidance for practical application of XAI in neuroimaging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。