提出轻量级医疗视觉语言持续学习框架,有效防止旧知识遗忘。
MedPEFT-CL: Dual-Phase Parameter-Efficient Continual Learning with Medical Semantic Adapter and Bidirectional Memory Consolidation
- 双阶段设计:先适应新解剖结构,再双向记忆巩固
- 仅新增参数占0.1%,却显著减少遗忘率
- 适合临床场景中需持续更新的医疗模型
医疗视觉语言分割模型在适应新解剖结构时易发生灾难性遗忘,需完全重训,限制其临床部署。尽管持续学习已被广泛研究,但针对医疗视觉语言任务的专门方法仍较少。本文提出MedPEFT-CL,一种基于CLIPSeg的参数高效持续学习框架,通过双阶段架构实现新任务高效学习与旧知识保留。第一阶段为自适应学习,基于语义相似性分配适配器并结合提示相似性分析进行参数高效微调;第二阶段为知识巩固,采用双向Fisher-记忆协调机制。该循环强化:巩固指导回放优先级,新任务提供挑战样本以优化保留策略。主要贡献包括:(1) 基于语义的适配器分配机制,提升新医疗任务学习效率;(2) 双模态LoRA适配,大幅降低可训练参数同时保持跨模态学习能力;(3) 双向Fisher-记忆协调,有效防止先前医疗任务的灾难性遗忘。在多个医疗数据集上的实验表明,该框架在极低参数开销下显著缓解遗忘并保持性能,适用于医疗视觉语言场景中的持续学习。
原文摘要 · Abstract (English)
Medical vision-language segmentation models suffer from catastrophic forgetting when adapting to new anatomical structures, requiring complete retraining that limits their clinical deployment. Although continual learning approaches have been studied for various applications, targeted research on continual learning approaches specifically designed for medical vision-language tasks remains underexplored. We propose MedPEFT-CL, a parameter-efficient continual learning framework that addresses both efficient learning of new tasks and preservation of previous knowledge through a dual-phase architecture based on CLIPSeg. Our dual-phase architecture features an adaptive learning phase that employs semantic similarity-based adapter allocation and parameter-efficient fine-tuning for medical tasks through prompt similarity analysis, and a knowledge consolidation phase employing bi-directional Fisher-memory coordination. This creates a reinforcing cycle: consolidation directs replay priorities while new tasks provide challenging samples that improve retention strategies. Our key contributions are: (1) a semantic-driven adapter allocation mechanism that enables efficient learning of new medical tasks, (2) a bi-modal LoRA adaptation that significantly reduces trainable parameters while maintaining cross-modal learning, and (3) bidirectional Fisher-memory coordination that prevents catastrophic forgetting from previous medical tasks. Extensive experiments across diverse medical datasets demonstrate superior forgetting mitigation and performance retention with minimal parameter overhead, making the framework effective for continual learning in medical vision-language scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。