提出多尺度全局-实例提示调优,提升医学图像分割在持续域变化下的适应能力。
Multi-Scale Global-Instance Prompt Tuning for Continual Test-time Adaptation in Medical Image Segmentation
- 设计自适应尺度实例提示与多尺度全局提示,分层捕捉局部与全局知识。
- 在多个医学图像分割数据集上优于现有方法,实现持续域变化下的稳定性能。
- 适合长期部署于跨中心医疗影像的模型在线适应场景。
临床中心获取的医学图像常存在分布偏移,严重阻碍预训练语义分割模型在多域真实场景中的应用。持续测试时自适应(CTTA)成为应对跨域偏移的有前景方案。现有方法依赖增量更新模型参数,易导致误差累积和灾难性遗忘,尤其在长期适应中。近期基于提示调优的方法通过仅更新视觉提示缓解上述问题,但仍存在三方面局限:1)提示缺乏多尺度多样性;2)未充分融合实例级知识;3)存在隐私泄露风险。为此,本文提出多尺度全局-实例提示调优(MGIPT),增强提示的尺度多样性,同时捕获全局与实例级知识以实现鲁棒的持续测试时自适应。MGIPT包含自适应尺度实例提示(AIP)和多尺度全局提示(MGP)。AIP动态学习轻量级、实例特定的提示,并通过自适应最优尺度选择机制减轻误差累积。MGP在不同尺度上捕捉领域级知识,具备抗遗忘能力。两者通过加权集成结合,实现双层次信息融合的高效适应。大量实验表明,MGIPT在多个医学图像分割基准上超越当前最优方法,在持续变化的目标域中表现稳健。
原文摘要 · Abstract (English)
Distribution shift is a common challenge in medical images obtained from different clinical centers, significantly hindering the deployment of pre-trained semantic segmentation models in real-world applications across multiple domains. Continual Test-Time Adaptation(CTTA) has emerged as a promising approach to address cross-domain shifts during continually evolving target domains. Most existing CTTA methods rely on incrementally updating model parameters, which inevitably suffer from error accumulation and catastrophic forgetting, especially in long-term adaptation. Recent prompt-tuning-based works have shown potential to mitigate the two issues above by updating only visual prompts. While these approaches have demonstrated promising performance, several limitations remain:1)lacking multi-scale prompt diversity, 2)inadequate incorporation of instance-specific knowledge, and 3)risk of privacy leakage. To overcome these limitations, we propose Multi-scale Global-Instance Prompt Tuning(MGIPT), to enhance scale diversity of prompts and capture both global- and instance-level knowledge for robust CTTA. Specifically, MGIPT consists of an Adaptive-scale Instance Prompt(AIP) and a Multi-scale Global-level Prompt(MGP). AIP dynamically learns lightweight and instance-specific prompts to mitigate error accumulation with adaptive optimal-scale selection mechanism. MGP captures domain-level knowledge across different scales to ensure robust adaptation with anti-forgetting capabilities. These complementary components are combined through a weighted ensemble approach, enabling effective dual-level adaptation that integrates both global and local information. Extensive experiments on medical image segmentation benchmarks demonstrate that our MGIPT outperforms state-of-the-art methods, achieving robust adaptation across continually changing target domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。