arXiv:2606.23487cs.AI2026-06

提出CADRE框架,让医疗视觉语言模型持续学习时既不遗忘旧知识也不偏离可信先验。

CADRE: Stable, Parameter Efficient Adaptation of Medical Vision Language Models with Bounded Forgetting and Prior Drift

  • 用低秩适配结合自适应弹性权重巩固,控制遗忘与偏移
  • 仅更新0.23%参数,遗忘率降为基线的七分之一(0.011对0.075)
  • 适合临床部署中需长期稳定更新的医疗AI系统

医学视觉语言模型(如BiomedCLIP)泛化能力强,但将其适配至新临床服务时,安全问题不亚于准确率问题。更新已部署模型以应对新影像模态可能无声失效:既会遗忘已有模态(灾难性遗忘),又会偏离可信预训练先验,转向模态特异性捷径。本文聚焦参数高效持续适配中的这两类风险,提出CADRE——一种冻结主干的框架,结合低秩适配(LoRA)与在线自缩放、相似性感知的弹性权重巩固项,约束保留能力损失;同时引入锚点-先验惩罚项,限制嵌入向量从冻结先验漂移。两个理论保证(总巩固质量有界、尺度不变性)消除了传统EWC的尺度敏感性缺陷。以乳腺癌在三类差异极大的模态(组织病理、超声、胸部X光)上的跨模态压力测试为例,在多种子、多顺序协议下,采用成对显著性检验,仅训练约0.23%参数,CADRE在准确率、SPQ、反向迁移上均最优,遗忘最低,相比最强正则化基线遗忘率降低约七倍(0.075→0.011,配对p=0.023),且实现正向反向迁移,所有基线均为负。这些特性被定义为符合临床安全需求的稳定性指标,而非部署保障;分布偏移与对抗输入鲁棒性不在研究范围。

原文摘要 · Abstract (English)

Medical vision-language models (VLMs) such as BiomedCLIP generalize broadly, but adapting them to a clinical service is as much a safety problem as an accuracy one. Updating a deployed model for a new imaging modality can fail silently in two ways that harm patients: it can forget modalities it already handled (catastrophic forgetting), and it can drift from its trustworthy pretrained prior toward modality-specific shortcuts. We study parameter-efficient continual adaptation through these two properties rather than leaderboard accuracy, presenting CADRE: a frozen-backbone framework combining low-rank adaptation (LoRA) with an online, self-scaling, similarity-aware elastic weight consolidation term that bounds retained-competence loss, and an anchor-to-prior penalty bounding embedding drift from the frozen prior. Two short guarantees, a bound on total consolidation mass and a scale-invariance property, remove the scale-related sources of vanilla EWC's order fragility. Using breast cancer across three maximally dissimilar modalities (histopathology, ultrasound, chest radiography) as a controlled cross-modality stress test, under a multi-seed, multi-order protocol with paired significance testing and training approximately 0.23% of parameters, CADRE attains the highest accuracy, SPQ, and backward transfer and the lowest forgetting among adapting methods, reducing forgetting roughly sevenfold versus the strongest regularized baseline (0.075 to 0.011; paired p=0.023) and achieving positive backward transfer where every baseline is negative. We frame these as stability properties aligned with clinical-safety desiderata, not a deployment guarantee; robustness to distribution shift and adversarial inputs is out of scope.

医疗AI持续学习参数效率模型稳定

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。