用提示词聚类提升病理切片分析的准确率与可解释性
PTCMIL: Multiple Instance Learning via Prompt Token Clustering for Whole Slide Image Analysis
- 引入可学习提示词，在统一框架中联合聚类与分类任务
- 在8个数据集上超越现有方法，生存分析任务表现最优
- 适合医学图像分析、需要高可解释性的研究者使用
多重实例学习（MIL）推动了全切片图像（WSI）分析的发展，但面对切片的复杂性和异质性仍面临挑战。现有MIL方法难以将多样化的图像块信息整合为稳健的切片表示。尽管视觉变压器（ViT）和基于聚类的方法展现潜力，但计算开销大且无法捕捉任务特异性和切片特异性变化。为此，我们提出PTCMIL，一种基于提示词聚类的新型ViT用于MIL聚合。通过在ViT主干中引入可学习提示词，PTCMIL以端到端方式统一聚类与预测任务，动态对齐聚类结果与下游任务，采用投影式聚类适应每张切片，降低复杂度同时保留图像块异质性。结合令牌合并与原型池化策略，高效捕获任务相关模式。在8个数据集上的大量实验表明，其在分类与生存分析任务中均优于当前最佳方法。系统消融研究验证了其鲁棒性与强可解释性。代码已开源：https://github.com/ubc-tea/PTCMIL。
原文摘要 · Abstract (English)
Multiple Instance Learning (MIL) has advanced WSI analysis but struggles with the complexity and heterogeneity of WSIs. Existing MIL methods face challenges in aggregating diverse patch information into robust WSI representations. While ViTs and clustering-based approaches show promise, they are computationally intensive and fail to capture task-specific and slide-specific variability. To address these limitations, we propose PTCMIL, a novel Prompt Token Clustering-based ViT for MIL aggregation. By introducing learnable prompt tokens into the ViT backbone, PTCMIL unifies clustering and prediction tasks in an end-to-end manner. It dynamically aligns clustering with downstream tasks, using projection-based clustering tailored to each WSI, reducing complexity while preserving patch heterogeneity. Through token merging and prototype-based pooling, PTCMIL efficiently captures task-relevant patterns. Extensive experiments on eight datasets demonstrate its superior performance in classification and survival analysis tasks, outperforming state-of-the-art methods. Systematic ablation studies confirm its robustness and strong interpretability. The code is released at https://github.com/ubc-tea/PTCMIL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。