通过自适应融合多个病理模型,提升诊断准确率并揭示模型专长。
AdaFusion: Prompt-Guided Inference with Adaptive Fusion of Pathology Foundation Models
- 根据组织表型动态融合多模型特征,实现智能推理
- 在三个真实任务中均超越单个模型表现,提升性能
- 可解释各模型的生物语义专长,适合临床辅助决策
病理基础模型(PFMs)通过大规模未标注组织病理图像的自监督预训练展现出强大的表征能力。然而,其多样且不透明的预训练背景——由数据、结构及训练因素共同塑造——引入潜在偏差,影响下游应用的泛化性与可解释性。本文提出AdaFusion,一种新型提示引导推理框架,据我们所知,是首个动态集成多个PFM互补知识的方法。该方法压缩并对齐来自不同模型的切片级特征,采用轻量级注意力机制,基于组织表型上下文自适应融合。我们在三个真实世界基准上评估:治疗反应预测、肿瘤分级与空间基因表达推断。结果表明,该方法在分类与回归任务中持续优于单一PFM,同时提供对各模型生物语义专长的可解释洞察。这些成果凸显了AdaFusion在弥合异构PFM方面的潜力,实现性能提升与模型归纳偏置可解释性的双重优势。
原文摘要 · Abstract (English)
Pathology foundation models (PFMs) have demonstrated strong representational capabilities through self-supervised pre-training on large-scale, unannotated histopathology image datasets. However, their diverse yet opaque pretraining contexts, shaped by both data-related and structural/training factors, introduce latent biases that hinder generalisability and transparency in downstream applications. In this paper, we propose AdaFusion, a novel prompt-guided inference framework that, to our knowledge, is among the very first to dynamically integrate complementary knowledge from multiple PFMs. Our method compresses and aligns tile-level features from diverse models and employs a lightweight attention mechanism to adaptively fuse them based on tissue phenotype context. We evaluate AdaFusion on three real-world benchmarks spanning treatment response prediction, tumour grading, and spatial gene expression inference. Our approach consistently surpasses individual PFMs across both classification and regression tasks, while offering interpretable insights into each model's biosemantic specialisation. These results highlight AdaFusion's ability to bridge heterogeneous PFMs, achieving both enhanced performance and interpretability of model-specific inductive biases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。