用提示对齐实现病理图像终身学习,避免遗忘且无需重训。
MergeSlide: Continual Model Merging and Task-to-Class Prompt-Aligned Inference for Lifelong Learning on Whole Slide Images
- 新任务用类别提示定义,轻量微调后正交合并到统一模型。
- 在6个TCGA数据集流上准确率超回放方法和零样本基线。
- 适合需要持续学习的医学图像分析场景,如癌症诊断迭代升级。
全切片图像(WSI)上的终身学习旨在顺序训练或微调统一模型以应对多种癌症相关任务,降低数据传输与处理资源消耗,尤其针对千兆级规模的WSI。本文提出MergeSlide框架,将终身学习视为模型融合问题,利用视觉-语言病理学基础模型。新任务到来时:1)使用类别感知提示定义;2)采用无MLP主干微调数个周期;3)通过正交持续融合策略合并至统一模型,保留性能并缓解灾难性遗忘。针对任务身份未知的类别增量学习(CLASS-IL)推理,引入任务-类别提示对齐(TCP)推理:先用任务级提示识别最相关任务,再应用对应类别提示生成预测。我们在六个TCGA数据集流上进行实验,结果表明MergeSlide优于基于回放的持续学习方法及视觉-语言零样本基线。代码与数据见https://github.com/caodoanh2001/MergeSlide。
原文摘要 · Abstract (English)
Lifelong learning on Whole Slide Images (WSIs) aims to train or fine-tune a unified model sequentially on cancer-related tasks, reducing the resources and effort required for data transfer and processing, especially given the gigabyte-scale size of WSIs. In this paper, we introduce MergeSlide, a simple yet effective framework that treats lifelong learning as a model merging problem by leveraging a vision-language pathology foundation model. When a new task arrives, it is: 1) defined with class-aware prompts, 2) fine-tuned for a few epochs using an MLP-free backbone, and 3) merged into a unified model using an orthogonal continual merging strategy that preserves performance and mitigates catastrophic forgetting. For inference under the class-incremental learning (CLASS-IL) setting, where task identity is unknown, we introduce Task-to-Class Prompt-aligned (TCP) inference. Specifically, TCP first identifies the most relevant task using task-level prompts and then applies the corresponding class-aware prompts to generate predictions. To evaluate MergeSlide, we conduct experiments on a stream of six TCGA datasets. The results show that MergeSlide outperforms both rehearsal-based continual learning and vision-language zero-shot baselines. Code and data are available at https://github.com/caodoanh2001/MergeSlide.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。