用稀疏自编码器解析乳腺影像模型如何学习关键病变概念
Mammo-SAE: Interpreting Breast Cancer Concept Learning with Sparse Autoencoders
- 通过稀疏自编码器分析乳腺影像模型的潜在特征
- 发现高激活神经元常对应真实病灶区域，揭示模型决策依据
- 适合医学AI可解释性研究者和放射科医生参考
在医疗影像等高风险领域，模型可解释性对临床应用至关重要。本文将稀疏自编码器（SAE）方法引入乳腺影像分析，基于大规模乳腺钼靶图像-报告对预训练的 {Mammo-CLIP} 模型，构建局部补丁级 exttt{Mammo-SAE}，以识别与临床相关概念（如肿块、可疑钙化）相关的潜在特征。结果表明，SAE隐空间中最高激活的类别级神经元通常与真实病灶区域对齐，并发现了若干影响模型决策的混淆因素。此外，我们还分析了微调阶段模型依赖的潜在神经元，以提升乳腺病变预测性能。该研究展示了可解释的SAE隐表示在深入理解基础模型各层内部机制方面的潜力。代码将在 https://krishnakanthnakka.github.io/MammoSAE/ 公开。
原文摘要 · Abstract (English)
Interpretability is critical in high-stakes domains such as medical imaging, where understanding model decisions is essential for clinical adoption. In this work, we introduce Sparse Autoencoder (SAE)-based interpretability to breast imaging by analyzing {Mammo-CLIP}, a vision--language foundation model pretrained on large-scale mammogram image--report pairs. We train a patch-level \texttt{Mammo-SAE} on Mammo-CLIP to identify and probe latent features associated with clinically relevant breast concepts such as \textit{mass} and \textit{suspicious calcification}. Our findings reveal that top activated class level latent neurons in the SAE latent space often tend to align with ground truth regions, and also uncover several confounding factors influencing the model's decision-making process. Additionally, we analyze which latent neurons the model relies on during downstream finetuning for improving the breast concept prediction. This study highlights the promise of interpretable SAE latent representations in providing deeper insight into the internal workings of foundation models at every layer for breast imaging. The code will be released at https://krishnakanthnakka.github.io/MammoSAE/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。