VIViT让MRI分割模型适应不同扫描参数，提升实际应用效果。
VIViT: Variable-Input Vision Transformer Framework for 3D MR Image Segmentation
- 设计可变输入的ViT框架，支持不同组合的MRI对比度输入
- 脑梗死和脑肿瘤分割分别达到0.624和0.883的Dice分数
- 适合处理真实世界中多变扫描协议的医学图像任务
自监督预训练技术被广泛用于提升下游任务性能。然而，现实中的磁共振（MR）研究通常因采集协议差异而包含不同组合的对比度，这对当前深度学习方法在大规模预训练及不同输入需求的下游任务上带来挑战，因为这些方法通常要求固定的输入模态或对比度。为解决此问题，我们提出可变输入视觉变换器（VIViT），一种基于Transformer的框架，专为具有不同对比度组合的研究进行自监督预训练与分割微调。该能力使我们的方法最大化预训练数据利用率，并能将学到的知识迁移到下游任务，即使输入要求存在差异。我们在脑梗死和脑肿瘤分割任务上验证了该方法，其平均Dice分数分别达到0.624和0.883，优于现有CNN与ViT基线模型。结果表明，该设计在异构真实MR数据上的适应性与性能更优。
原文摘要 · Abstract (English)
Self-supervised pretrain techniques have been widely used to improve the downstream tasks' performance. However, real-world magnetic resonance (MR) studies usually consist of different sets of contrasts due to different acquisition protocols, which poses challenges for the current deep learning methods on large-scale pretrain and different downstream tasks with different input requirements, since these methods typically require a fixed set of input modalities or, contrasts. To address this challenge, we propose variable-input ViT (VIViT), a transformer-based framework designed for self-supervised pretraining and segmentation finetuning for variable contrasts in each study. With this ability, our approach can maximize the data availability in pretrain, and can transfer the learned knowledge from pretrain to downstream tasks despite variations in input requirements. We validate our method on brain infarct and brain tumor segmentation, where our method outperforms current CNN and ViT-based models with a mean Dice score of 0.624 and 0.883 respectively. These results highlight the efficacy of our design for better adaptability and performance on tasks with real-world heterogeneous MR data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。