用多视角读出法让冻结的DINOv3模型高效完成医学分割。
DINO-MVR: Multi-View Readout of Frozen DINOv3 for Annotation-Efficient Medical Segmentation

- 仅训练轻量MLP探针读取冻结DINOv3特征,不更新主干网络。
- 在少量标注下达到接近全量标注的性能,如5例标注即达98.4%效果。
- 适合数据稀缺场景,尤其适用于内窥镜、皮肤镜和MRI分割任务。
将基础模型应用于医学分割通常需要微调主干或使用高容量解码器,但在标注稀少时难以可靠实现。我们发现冻结的DINOv3特征已包含有用的结构与边界信息,主要瓶颈在于特征读出方式。为此提出DINO-MVR:一种仅在冻结DINOv3最后三个Transformer块特征上训练轻量MLP探针的多视角读出框架。推理时通过多分辨率与测试时增强生成概率图,采用熵加权融合并结合简单空间正则化进行优化;对三维输入进一步使用高斯z轴平滑以提升切片间一致性。在内窥镜、皮肤镜和MRI基准上固定评估协议下,DINO-MVR表现优异,包括Kvasir-SEG上0.895 Dice、ISIC 2018上0.897 Dice、BraTS FLAIR全肿瘤分割上0.908 Dice。仅用5例标注的BraTS数据,即恢复参考运行(40例)98.4%的性能。结果表明,配合有效多视角读出,冻结自监督视觉主干可支持高精度医学分割。
原文摘要 · Abstract (English)
Adapting foundation models to medical segmentation typically requires either backbone fine-tuning or high-capacity task-specific decoders, both of which are difficult to fit reliably when annotations are scarce. We show that frozen DINOv3 features already contain useful structural and boundary cues for medical segmentation, and that the main bottleneck lies in how these features are read out. We propose DINO-MVR, a Multi-View Readout framework for annotation-efficient medical segmentation. DINO-MVR trains only lightweight MLP probes on features from the final three transformer blocks of a frozen DINOv3 backbone, without updating the backbone itself. At inference, each input is interpreted through complementary resolutions and test-time augmentations, whose probability maps are combined by entropy-weighted fusion and refined with simple spatial regularization. For volumetric inputs, Gaussian z-axis smoothing further improves inter-slice consistency. Under fixed evaluation protocols on endoscopy, dermoscopy, and MRI benchmarks, DINO-MVR achieves strong readout-only performance, including 0.895 Dice on Kvasir-SEG, 0.897 Dice on ISIC 2018, and 0.908 Dice on BraTS FLAIR whole-tumor segmentation. With only five annotated BraTS patients, it recovers 98.4% of the performance obtained by the 40-patient BraTS reference run. These results suggest that frozen self-supervised vision backbones can support accurate medical segmentation when paired with an effective multi-view readout.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。