arXiv:2411.09894cs.CV2024-11NeurIPS被引 15

用概念引导增强特征,让病理模型更懂特定癌症任务。

Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancement

  • 基于专家设计的概念锚点,动态优化基础模型特征
  • 在多个公开数据集上显著提升分类性能与泛化能力
  • 适合需要可解释性病理分析的临床研究者使用

全切片图像(WSI)分析在医学影像领域日益重要。近年来,病理基础模型已展现出从WSI中提取强大特征表示的能力。然而,这些基础模型通常面向通用病理图像分析,对特定下游任务或癌种可能不够优化。本文提出概念锚点引导的任务特异性特征增强(CATE)框架,通过任务相关的概念信息动态校准基础模型的通用特征。基于病理视觉-语言模型结合专家提示生成的任务概念,设计了两个互连模块:概念引导的信息瓶颈模块通过最大化图像特征与概念锚点间的互信息,强化任务相关特征并抑制冗余信息;概念-特征干扰模块则利用校准后特征与概念锚点的相似性,进一步生成具有判别力的任务特异性特征。在多个公开的WSI数据集上的大量实验表明,CATE显著提升了MIL模型的性能与泛化能力。热图和Umap可视化结果也验证了CATE的有效性与可解释性。源代码见https://github.com/HKU-MedAI/CATE。

原文摘要 · Abstract (English)

Whole slide image (WSI) analysis is gaining prominence within the medical imaging field. Recent advances in pathology foundation models have shown the potential to extract powerful feature representations from WSIs for downstream tasks. However, these foundation models are usually designed for general-purpose pathology image analysis and may not be optimal for specific downstream tasks or cancer types. In this work, we present Concept Anchor-guided Task-specific Feature Enhancement (CATE), an adaptable paradigm that can boost the expressivity and discriminativeness of pathology foundation models for specific downstream tasks. Based on a set of task-specific concepts derived from the pathology vision-language model with expert-designed prompts, we introduce two interconnected modules to dynamically calibrate the generic image features extracted by foundation models for certain tasks or cancer types. Specifically, we design a Concept-guided Information Bottleneck module to enhance task-relevant characteristics by maximizing the mutual information between image features and concept anchors while suppressing superfluous information. Moreover, a Concept-Feature Interference module is proposed to utilize the similarity between calibrated features and concept anchors to further generate discriminative task-specific features. The extensive experiments on public WSI datasets demonstrate that CATE significantly enhances the performance and generalizability of MIL models. Additionally, heatmap and umap visualization results also reveal the effectiveness and interpretability of CATE. The source code is available at https://github.com/HKU-MedAI/CATE.

病理分析特征增强可解释性基础模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。