用可视化方法解析病理模型如何看懂癌细胞，让黑箱变透明。
Class Visualizations and Activation Atlases for Enhancing Interpretability in Deep Learning-Based Computational Pathology
- 开发可视化框架，分析Transformer模型在病理图像中的特征表示
- 发现不同层级分类中专家一致性从0.75降至0.11，体现病理复杂性
- 激活图揭示层次化结构，适合临床医生和研究者验证模型决策
基于Transformer的模型在计算病理学中快速普及，用于从H&E全片图像预测分子与临床生物标志物，但可解释性未能跟上模型复杂度。尽管已有归因和生成类方法，但类别可视化（CVs）和激活图（AAs）尚未系统评估。本研究构建可视化框架，评估了跨组织癌症分类任务中不同标签粒度下的CVs与AAs表现。四位病理科医生对真实与生成图像进行标注，量化观察者间一致性，辅以归因与相似性指标。结果显示，CVs在形态差异明显组织中保持可识别性，但在重叠癌亚型中区分度下降；组织分类中一致性从扫描图的Fleiss k=0.75降至CV图的k=0.31，癌亚型任务中趋势类似。激活图显示层依赖结构：粗粒度组织概念形成连贯区域，细粒度亚型则分散且重叠。组织分类一致性为k=0.58，粗粒度分组为k=0.82，亚型级别仅为k=0.11。地图可分性与专家对真实图像的一致性高度相关，表明表征模糊性反映了病理本身的复杂性。归因指标在低复杂度场景下近似专家变异，而感知与分布度量一致性较差。总体而言，概念级可视化揭示了模型中结构化的形态流形，为跨标签粒度的专家导向分析提供框架。
原文摘要 · Abstract (English)
The rapid adoption of transformer-based models in computational pathology has enabled prediction of molecular and clinical biomarkers from H&E whole-slide images, yet interpretability has not kept pace with model complexity. While attribution- and generative-based methods are common, feature visualization approaches such as class visualizations (CVs) and activation atlases (AAs) have not been systematically evaluated for these models. We developed a visualization framework and assessed CVs and AAs for a transformer-based foundation model across tissue and multi-organ cancer classification tasks with increasing label granularity. Four pathologists annotated real and generated images to quantify inter-observer agreement, complemented by attribution and similarity metrics. CVs preserved recognizability for morphologically distinct tissues but showed reduced separability for overlapping cancer subclasses. In tissue classification, agreement decreased from Fleiss k = 0.75 (scans) to k = 0.31 (CVs), with similar trends in cancer subclass tasks. AAs revealed layer-dependent organization: coarse tissue-level concepts formed coherent regions, whereas finer subclasses exhibited dispersion and overlap. Agreement was moderate for tissue classification (k = 0.58), high for coarse cancer groupings (k = 0.82), and low at subclass level (k = 0.11). Atlas separability closely tracked expert agreement on real images, indicating that representational ambiguity reflects intrinsic pathological complexity. Attribution-based metrics approximated expert variability in low-complexity settings, whereas perceptual and distributional metrics showed limited alignment. Overall, concept-level feature visualization reveals structured morphological manifolds in transformer-based pathology models and provides a framework for expert-centered interrogation of learned representations across label granularities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。