arXiv:2410.04689cs.CV2024-10中稿 · Medical Image Comp…被引 4

轻量级微调模型可逐步分割人体器官,避免遗忘旧知识且节省存储。

Low-Rank Continual Pyramid Vision Transformer: Incrementally Segment Whole-Body Organs in CT with Light-Weighted Adaptation

  • 在冻结主干网络基础上,仅对关键层添加低秩适配参数。
  • 在121个器官上持续学习,精度接近主流模型上限。
  • 适合医疗场景中隐私敏感、数据不可复用的增量学习需求。

深度分割网络在特定数据集上表现优异,但在临床实践中,常需将预训练模型动态扩展以分割新器官,而无需访问历史数据或从头训练,这能有效应对患者隐私和存储问题。该过程可视为持续语义分割(CSS)问题。现有方法或面临灾难性遗忘,或导致不可接受的内存开销。本文提出一种基于低秩适配(LoRA)的轻量化持续全身体积器官分割模型。首先在初始任务上训练并冻结金字塔视觉变换器(PVT)主干模型,随后为每个新增任务仅在关键层(即图像块嵌入、多头注意力和前馈层)添加轻量级可训练参数。通过系统性架构分析,确认三类层对任务适应至关重要,同时保持绝大部分参数不变。在覆盖121个器官的四个数据集上持续训练与测试,结果表明模型在不发生灾难性遗忘的同时,参数增长极低,分割精度接近PVT与nnUNet的上限,显著优于其他正则化型CSS方法;相较领先架构型方法,参数增长率更低,性能相当。

原文摘要 · Abstract (English)

Deep segmentation networks achieve high performance when trained on specific datasets. However, in clinical practice, it is often desirable that pretrained segmentation models can be dynamically extended to enable segmenting new organs without access to previous training datasets or without training from scratch. This would ensure a much more efficient model development and deployment paradigm accounting for the patient privacy and data storage issues. This clinically preferred process can be viewed as a continual semantic segmentation (CSS) problem. Previous CSS works would either experience catastrophic forgetting or lead to unaffordable memory costs as models expand. In this work, we propose a new continual whole-body organ segmentation model with light-weighted low-rank adaptation (LoRA). We first train and freeze a pyramid vision transformer (PVT) base segmentation model on the initial task, then continually add light-weighted trainable LoRA parameters to the frozen model for each new learning task. Through a holistically exploration of the architecture modification, we identify three most important layers (i.e., patch-embedding, multi-head attention and feed forward layers) that are critical in adapting to the new segmentation tasks, while retaining the majority of the pretrained parameters fixed. Our proposed model continually segments new organs without catastrophic forgetting and meanwhile maintaining a low parameter increasing rate. Continually trained and tested on four datasets covering different body parts of a total of 121 organs, results show that our model achieves high segmentation accuracy, closely reaching the PVT and nnUNet upper bounds, and significantly outperforms other regularization-based CSS methods. When comparing to the leading architecture-based CSS method, our model has a substantial lower parameter increasing rate while achieving comparable performance.

医学图像持续学习轻量化分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。