arXiv:2605.13798cs.CV2026-05被引 1

无需训练即可生成跨模态3D医学图像的稳定体素特征,支持直接对应查询。

VoxCor: Training-Free Volumetric Features for Multimodal Voxel Correspondence

论文配图:VoxCor: Training-Free Volumetric Features for Multimodal Voxel Correspondence
图 1 · 摘自论文原文
  • 用三平面视觉变换器+加权偏最小二乘法,从冻结的2D模型提取可复用的3D特征
  • 在腹部MR-CT和脑部T2w-T1w任务中,跨受试者跨模态匹配性能显著提升
  • 适合需要快速部署、无需微调的多模态医学图像分析场景

跨模态3D医学图像分析需要在不同成像对比度、扫描仪和采集协议下保持体素级表示的解剖一致性。现有方法利用冻结的2D视觉变换器(ViT)基础模型生成此类表示,但通常仅沿单一解剖轴提取特征,并在配准求解器中逐对适配,导致其他视角信息被浪费,且特征难以迁移至新体积。本文提出VoxCor,一种无需训练的拟合-转换方法,可从冻结的2D ViT基础模型中获取可复用的体素特征表示。离线拟合阶段,VoxCor结合三平面ViT推理与紧凑的闭式加权偏最小二乘(WPLS)投影,利用拟合时的体素对应关系,在三平面特征空间中选择模态稳定的解剖方向。转换阶段,新体积仅通过三平面ViT推理和线性投影即可映射,无需微调或配准。体素对应关系可通过最近邻搜索直接查询。我们在同体受试者腹部MR-CT和异体受试者HCP T2w-T1w任务上评估了VoxCor,采用可变形配准、体素级k近邻分割和分割中心地标定位。结果表明,VoxCor在最困难的跨受试者、跨模态迁移设置中表现更优,降低了编码器对密集对应传输的敏感性,并达到与手工设计描述符和学习型3D特征相当的配准性能。这使VoxCor成为超越成对配准的下游多模态分析可复用特征层。代码、配置文件和实现细节已公开于GitHub:https://github.com/guneytombak/VoxCor。

原文摘要 · Abstract (English)

Cross-modal 3D medical image analysis requires voxelwise representations that remain anatomically consistent across imaging contrasts, scanners, and acquisition protocols. Recent work has shown that frozen 2D Vision Transformer (ViT) foundation models can support such representations, but typical pipelines extract features along a single anatomical axis and adapt those features inside a registration solver for one image pair at a time, leaving complementary viewing directions unused and producing representations that do not transfer to new volumes. We introduce VoxCor, a training-free fit--transform method for reusable volumetric feature representations from frozen 2D ViT foundation models. During an offline fitting phase, VoxCor combines triplanar ViT inference with a compact closed-form weighted partial least squares (WPLS) projection that uses fitting-time voxel correspondences to select modality-stable anatomical directions in the triplanar feature space. At transform time, new volumes are mapped by triplanar ViT inference and linear projection alone, without fine-tuning or registration. Voxel correspondences can then be queried directly by nearest-neighbor search. We evaluate VoxCor on intra-subject Abdomen MR--CT and inter-subject HCP T2w--T1w tasks using deformable registration, voxelwise k-nearest-neighbor segmentation, and segmentation-center landmark localization. VoxCor improves the hardest cross-subject, cross-modality transfer settings, reduces encoder sensitivity for dense correspondence transfer, and yields registration performance competitive with handcrafted descriptors and learned 3D features. This positions VoxCor as a reusable feature layer for downstream multimodal analysis beyond pairwise registration. Code, configuration files, and implementation details are publicly available on GitHub at \href{https://github.com/guneytombak/VoxCor}{guneytombak/VoxCor}.

3D医学图像跨模态匹配特征提取无需训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。