用稀疏自编码器解析血细胞图像,实现可解释的医学影像分析。
CytoSAE: Interpretable Cell Embeddings for Hematology
- 基于4万张血细胞图像训练稀疏自编码器,发现形态相关视觉概念。
- 在骨髓涂片等外域数据上仍能识别关键病理特征,专家验证有效。
- 可定位病灶区域并生成患者/疾病特异性概念,适合临床可解释性研究。
稀疏自编码器(SAEs)是解释基于Transformer的基座模型机制的有力工具。近期,SAEs也被应用于视觉领域,实现了对视觉概念的发现及其在Transformer中令牌的逐块归因。尽管医学影像领域涌现出越来越多的基座模型,但解释其推理过程的工具仍十分匮乏。本文展示了SAEs在血液学中的适用性,提出CytoSAE,一种在超过4万张外周血单细胞图像上训练的稀疏自编码器。CytoSAE能泛化至多样且跨域的数据集,包括骨髓细胞学,成功识别出经医学专家验证的形态学相关概念。此外,我们展示了在生成患者特异性和疾病特异性概念方面的应用,实现对典型病变细胞和局部细胞异常的逐像素检测。我们在患者级AML亚型分类任务中量化了这些概念的影响,结果显示其性能可媲美当前最优模型,同时提供亚细胞层面的可解释性。源代码与模型权重已公开于https://github.com/dynamical-inference/cytosae。
原文摘要 · Abstract (English)
Sparse autoencoders (SAEs) emerged as a promising tool for mechanistic interpretability of transformer-based foundation models. Very recently, SAEs were also adopted for the visual domain, enabling the discovery of visual concepts and their patch-wise attribution to tokens in the transformer model. While a growing number of foundation models emerged for medical imaging, tools for explaining their inferences are still lacking. In this work, we show the applicability of SAEs for hematology. We propose CytoSAE, a sparse autoencoder which is trained on over 40,000 peripheral blood single-cell images. CytoSAE generalizes to diverse and out-of-domain datasets, including bone marrow cytology, where it identifies morphologically relevant concepts which we validated with medical experts. Furthermore, we demonstrate scenarios in which CytoSAE can generate patient-specific and disease-specific concepts, enabling the detection of pathognomonic cells and localized cellular abnormalities at the patch level. We quantified the effect of concepts on a patient-level AML subtype classification task and show that CytoSAE concepts reach performance comparable to the state-of-the-art, while offering explainability on the sub-cellular level. Source code and model weights are available at https://github.com/dynamical-inference/cytosae.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。