arXiv:2512.13072cs.CV2025-12被引 2

用动态检索增强学习,让医学多模态模型持续学习不遗忘。

Forging a Dynamic Memory: Retrieval-Guided Continual Learning for Generalist Medical Foundation Models

  • 通过1800万条医学数据实时检索,动态引导模型微调。
  • 在跨模态迁移中保持细微特征,性能超越现有方法。
  • 适合需要持续更新的医疗AI系统研发人员使用。

多模态生物医学视觉-语言模型在持续学习(CL)领域具有巨大潜力,但面临核心挑战:如何在保留细粒度模态内特征的同时弥合不同模态间显著的领域差异。为此,我们提出一个完整框架。基于从PubMed科学论文构建的1800万条多模态、全面的医学检索数据库,首次将检索增强生成(RAG)引入持续学习。具体而言,采用多模态、多层次的RAG系统,通过动态、按需的知识检索为模型微调提供实时指导。在此基础上,提出一种动态知识蒸馏框架,通过动态调节参数空间重要性、蒸馏知识粒度和参考数据集分布,精准解决上述核心矛盾。为充分验证策略的临床价值,设计了更严格的医学通用任务增量学习(MGTIL)基准,该基准可同时评估模型对重大领域偏移的适应能力、细微领域内特征的保留能力以及对新型复杂医疗任务的实时学习能力。大量实验结果表明,所提方法在所有指标上均达到当前最优(SOTA)表现。代码已附于补充材料中。

原文摘要 · Abstract (English)

Multimodal biomedical Vision-Language Models (VLMs) exhibit immense potential in the field of Continual Learning (CL). However, they confront a core dilemma: how to preserve fine-grained intra-modality features while bridging the significant domain gap across different modalities. To address this challenge, we propose a comprehensive framework. Leveraging our 18-million multimodal and comprehensive medical retrieval database derived from PubMed scientific papers, we pioneer the integration of Retrieval-Augmented Generation (RAG) into CL. Specifically, we employ a multi-modal, multi-layer RAG system that provides real-time guidance for model fine-tuning through dynamic, on-demand knowledge retrieval. Building upon this, we introduce a dynamic knowledge distillation framework. This framework precisely resolves the aforementioned core dilemma by dynamically modulating the importance of the parameter space, the granularity of the distilled knowledge, and the data distribution of the reference dataset in accordance with the required level of detail. To thoroughly validate the clinical value of our strategy, we have designed a more rigorous \textbf{M}edical Generalist Task Incremental Learning (MGTIL) benchmark. This benchmark is engineered to simultaneously evaluate the model's capacity for adaptation to significant domain shifts, retention of subtle intra-domain features, and real-time learning of novel and complex medical tasks. Extensive experimental results demonstrate that our proposed method achieves state-of-the-art (SOTA) performance across all metrics. The code is provided in the supplementary materials.

持续学习医学AI检索增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。