用稀疏自编码器让医学图像模型的抽象表示变得可解释。
Sparse Autoencoders for Interpretable Medical Image Representation Learning
- 用稀疏自编码器学习医学图像的稀疏可解释特征。
- 仅用10个特征就恢复87.8%下游性能，维度压缩99.4%。
- 特征能对应临床概念，支持零样本语言检索。
视觉基础模型（FMs）在医学影像中表现优异，但其编码的隐变量表示抽象且不可解释。本研究探索使用稀疏自编码器（SAEs）替代这些黑箱表示，以获得人类可理解的稀疏特征。我们在909,873张来自TotalSegmentator数据集的CT和MRI二维切片上，对BiomedParse（生物医学）和DINOv3（通用）模型的嵌入进行SAE训练。结果表明：（a）学习到的稀疏特征能以高保真度重建原始嵌入（R²最高达0.941），仅用10个特征即可恢复87.8%的下游性能（维度压缩99.4%）；（b）在图像检索任务中保持语义一致性；（c）可通过大语言模型自动解释，对应具体临床概念；（d）实现零样本语言驱动的图像检索，连接临床语言与抽象表征。研究证明SAEs是构建可解释、概念驱动的医疗视觉系统的重要路径。代码仓库：https://github.com/pwesp/sail。
原文摘要 · Abstract (English)
Vision foundation models (FMs) achieve state-of-the-art performance in medical imaging. However, they encode information in abstract latent representations that clinicians cannot interrogate or verify. The goal of this study is to investigate Sparse Autoencoders (SAEs) for replacing opaque FM image representations with human-interpretable, sparse features. We train SAEs on embeddings from BiomedParse (biomedical) and DINOv3 (general-purpose) using 909,873 CT and MRI 2D image slices from the TotalSegmentator dataset. We find that learned sparse features: (a) reconstruct original embeddings with high fidelity (R2 up to 0.941) and recover up to 87.8% of downstream performance using only 10 features (99.4% dimensionality reduction), (b) preserve semantic fidelity in image retrieval tasks, (c) correspond to specific concepts that can be expressed in language using large language model (LLM)-based auto-interpretation. (d) bridge clinical language and abstract latent representations in zero-shot language-driven image retrieval. Our work indicates SAEs are a promising pathway towards interpretable, concept-driven medical vision systems. Code repository: https://github.com/pwesp/sail.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。