用ViT引导的UNet提升生物医学图像分割精度
Vision Transformer-Conditioned UNet for Domain-Adaptive Semantic Segmentation
- 将预训练ViT特征通过可学习令牌与UNet结合
- 在多模态医学影像上优于现有基线模型
- 适合需要高精度分割的生物医学研究者
语义分割在生物医学研究中对解剖特征分析至关重要，但视觉变换器（ViTs）在稀疏、细结构和低信噪比目标上的性能仍有差距。我们发现这部分源于提示式ViT模型中常用的轻量级像素解码器缺乏局部归纳偏置。为此提出ViTC-UNet，通过可学习令牌和双向注意力解码器，将冻结的预训练ViT表征条件化到UNet上。该方法融合了ViT的全局视觉先验与UNet的局部归纳偏置及高分辨率解码能力，且在跨域设置下无需端到端微调。ViTC-UNet在MRI和CT模态的语义分割任务中均超越基线表现，证明结构引导的UNet解码能高效适配大规模视觉先验至高复杂度生物医学分割任务。
原文摘要 · Abstract (English)
Semantic segmentation is essential for analysing anatomical features in biomedical research, yet a performance gap remains for Vision Transformers (ViTs) in the field, particularly for sparse, fine-structured, and low signal-to-noise targets. We attribute this challenge in part to the lightweight pixel decoders commonly used in promptable ViT models, who may lack the local inductive bias needed for high-precision biomedical masks. We bridge this gap by introducing ViTC-UNet, which conditions a UNet on frozen pre-trained ViT representations through learnable tokens and a two-way attention decoder. This combines ViT global visual priors with the local inductive bias and high-resolution decoding capacity of UNets, while avoiding end-to-end ViT fine-tuning even in cross-domain settings. ViTC-UNet outperforms baseline results in semantic segmentation tasks across MRI and CT modalities, demonstrating that structure-conditioned UNet decoding can efficiently adapt large-scale visual priors to high-complexity biomedical segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。