arXiv:2608.05808cs.CV2026-08

用文本锚定旧任务,解决医疗影像增量学习中的遗忘问题。

STAIL: Semantic Text-Anchored Incremental Learning for Medical Imaging via Large Language Models

论文配图:STAIL: Semantic Text-Anchored Incremental Learning for Medical Imaging via Large Language Models
图 1 · 摘自论文原文
  • 用少量图像+大量文本描述构建语义缓冲区,替代原始图像存储。
  • 在三个医学数据集上平均提升2.24%性能,减少3.55%遗忘率。
  • 适合需要持续学习、隐私敏感的医疗AI系统部署。

应用于医学图像分析的深度学习模型在动态环境中持续适应新临床任务时,常出现严重灾难性遗忘。主流增量学习方法通过重放历史原始图像缓解此问题,但像素级重放带来巨大存储开销,引发隐私担忧,且稀疏样本难以准确捕捉真实数据分布。受人类认知机制启发,我们提出一种新型框架——语义文本锚定增量学习(STAIL),用于连续临床任务。为突破重放瓶颈,STAIL引入非对称语义巩固缓冲区(SCB),仅需少量图像锚点与丰富文本描述,即可在极低存储成本下实现旧任务的密集语义重建。此外,设计基于大语言模型的语义锚定机制(LSAM),利用冻结大语言模型稳定的语义空间作为先验,显式将演化视觉特征锚定于文本表示,从宏观到微观层面引导和约束模型的可塑性与稳定性。在覆盖眼底、超声和X光影像的三个异构医学数据集上的广泛实验表明,STAIL作为即插即用模块表现优异,显著提升各类基线模型性能,平均实现AAA-AUC提升2.24%,BWT-AUC降低3.55%。代码已公开。

原文摘要 · Abstract (English)

Deep learning models applied to medical image analysis suffer from severe catastrophic forgetting when continually adapting to new clinical tasks in dynamic environments. Mainstream incremental learning methods typically mitigate this by rehearsing raw historical images. However, this pixel-level rehearsal incurs significant storage overhead, raises privacy concerns, and fails to adequately capture the true data distribution with sparse exemplars. Inspired by human cognitive mechanisms, we propose a novel framework termed Semantic Text-Anchored Incremental Learning (STAIL) for sequential clinical tasks. To overcome the rehearsal bottleneck, STAIL introduces an asymmetric semantic consolidation buffer (SCB). By incorporating a minimal set of image anchors and extensive textual descriptions, the SCB enables dense semantic reconstruction of old tasks at a minimal storage cost. Furthermore, we design an LLM-derived Semantic Anchoring Mechanism (LSAM) that leverages the stable semantic space of frozen large language models as developmental priors. This mechanism explicitly anchors evolving visual features to textual representations, guiding and constraining plasticity and stability at both macroscopic and microscopic levels. Extensive experiments across three heterogeneous medical datasets, covering fundus, ultrasound, and X-ray imaging, demonstrate that STAIL acts as a highly effective plug-and-play module. It comprehensively enhances the performance of various existing baselines, achieving average gains of 2.24\% in AAA-AUC for sustained performance and 3.55\% in BWT-AUC for reduced forgetting. Code is available.

增量学习医疗影像大模型语义锚定

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。