arXiv:2602.21735cs.CV2026-02被引 1

用旋转位置编码解决CT影像尺寸不一问题,提升文本与图像对齐精度。

SigVLP: Sigmoid Volume-Language Pre-Training for Self-Supervised CT-Volume Adaptive Representation Learning

  • 用旋转位置编码替代固定位置嵌入,支持任意尺寸的3D影像输入。
  • 通过分块采样和器官级文本标注,实现更精细的图文对齐,提升匹配精度。
  • 适用于多中心、异构设备采集的CT数据,适合医学影像自监督学习研究者。

大规模体积化医学影像数据通常来自不同厂商和设备,导致分辨率、切片厚度及每例扫描的切片数量差异显著。传统训练表示模型常需沿z轴裁剪或插值以获得固定尺寸块,不可避免造成信息损失。我们提出一种新训练方法:将体积视为3D块序列,采用旋转位置编码(Rotary Position Embedding),使z轴可作为无约束的时间维度处理。基于此,我们构建了新的视觉-语言模型SigVLP。在SigVLP中,旋转位置编码直接嵌入注意力操作,动态生成输入相关的正弦与余弦权重,确保查询与键投影的一致对齐,并适应任意输入尺寸。为支持可变输入大小训练,我们以分块方式采样计算机断层扫描(CT)体积,并配以局部器官级文本描述。相比使用完整报告进行条件建模,分块对齐提供更细粒度的监督,增强文本与体积表征间的关联,从而提高文本到体积的对齐精度。模型使用Muon优化器训练,并在多种下游任务上评估,包括零样本异常检测与器官分类、分割及检索任务。

原文摘要 · Abstract (English)

Large-scale, volumetric medical imaging datasets typically aggregate scans from different vendors and devices, resulting in highly variable resolution, slice thicknesses, and numbers of slices per study. Consequently, training representation models usually requires cropping or interpolating along the z-axis to obtain fixed-size blocks, which inevitably causes information loss. We propose a new training approach to overcome this limitation. Instead of absolute position embeddings, we interpret volumes as sequences of 3D chunks and adopt Rotary Position Embeddings, allowing us to treat the z-axis as an unconstrained temporal dimensions. Building on this idea, we introduce a new vision-language model: SigVLP. In SigVLP, we implement Rotary Position Embedding as the positional encoding method, which is applied directly within the attention operation, generating input-conditioned sine and cosine weights on the fly. This design ensures consistent alignment between query and key projections and adapts to any input sizes. To allow for variable input size during training, we sample Computed Tomography volumes in chunks and pair them with localized organ-wise textual observations. Compared to using entire reports for conditioning, chunkwise alignment provides finer-grained supervision, enabling the model to establish stronger correlations between the text and volume representations, thereby improving the precision of text-to-volume alignment. Our models are trained with the Muon optimizer and evaluated on a diverse set of downstream tasks, including zero-shot abnormality and organ classification, segmentation, and retrieval tasks.

医学影像自监督学习视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。