提出C²GR框架,缓解医学图像分割模型在持续学习中的遗忘问题。
C^2GR: Coupled Comprehensive Generative Replay for a Continually Learnable Universal Segmentation Model

- 通过联合扩散模型生成历史任务的图像-掩码对,保持结构真实性和语义一致性。
- 在20个跨模态任务上仅损失2.44%性能,接近联合训练效果。
- 适合需要隐私保护的医疗场景中持续更新通用分割模型的研究者。
通用分割模型在涉及不同成像模态和分割目标的任务中展现出巨大潜力。任务增量学习为在陆续到来的医疗部门数据上持续演化通用模型提供了一种保护隐私的途径。然而,仅在新任务上训练会导致对旧任务的遗忘,因为连续任务同时存在图像外观和分割目标的动态变化。为此,我们提出一种新型耦合全面生成回放(C²GR)框架,通过同步合成历史任务的图像-掩码对,缓解在外观与目标双重变化下的遗忘问题。该方法需保持图像-掩码对应关系以实现结构真实生成,并解决生成器与分割器异步优化的问题。具体地,我们提出贝叶斯联合扩散(BJD)方法,将对应关系建模为条件分布并通过条件去噪进行优化;进一步设计关系感知统一提示同步(RUPS)机制,通过共享的任务关系感知提示同步调节生成器与分割器。在涵盖20个不同模态与目标的任务上实验表明,C²GR相比使用全部任务数据的联合训练仅性能下降2.44%,有效缓解了由并发变化引发的遗忘。代码将公开于 https://github.com/mar-cry/C2GR。
原文摘要 · Abstract (English)
Universal segmentation models exhibit significant potential for diverse tasks involving different imaging modalities and segmentation objectives. Task-Incremental Learning provides a privacy-preserving approach to continually evolve a universal model on tasks from sequentially-arriving medical departments. However, training the model solely on the incoming task induces forgetting on past tasks, since consecutive tasks exhibit concurrent shifts in image appearance and segmentation objective. To address this problem, we propose a novel Coupled Comprehensive Generative Replay (C^2GR) framework that simultaneously synthesizes image-mask pairs of previous tasks to mitigate forgetting under concurrent appearance and objective shifts. This requires preserving image-mask correspondence for structure-realistic generation and bridging asynchronous optimization of the generator and segmentor for segmentation-oriented generation. Specifically, we propose a Bayesian Joint Diffusion (BJD) method that formulates the correspondence as conditional distributions optimized via conditional denoising. Furthermore, we develop a Relation-aware Unified Prompt Synchronization (RUPS) scheme to simultaneously modulate the generator and segmentor via a shared task-relation-aware prompt for synchronizing their optimization. Experiments on 20 tasks spanning diverse modalities and objectives demonstrate that C^2GR exhibits only a 2.44% drop in overall performance compared to joint training with all task data, effectively alleviating forgetting from the concurrent shifts. Our code will be made publicly available at https://github.com/mar-cry/C2GR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。