arXiv:2604.13561cs.CVcs.AI2026-04

探索3D腹部CT图文对齐中批次构成与数据量的影响,发现随机采样优于刻意平衡。

CLIP Architecture for Abdominal CT Image-Text Alignment and Zero-Shot Learning: Investigating Batch Composition and Data Scaling

论文配图:CLIP Architecture for Abdominal CT Image-Text Alignment and Zero-Shot Learning: Investigating Batch Composition and Data Scaling
图 1 · 摘自论文原文
  • 采用分段均衡采样控制批次中正常与异常比例,对比不同配比效果。
  • 在4362份数据上训练,性能随数据量增加呈亚线性增长,100%数据达71.88%。
  • 小批量下刻意平衡反而降低效果,随机采样更具正则化优势。

基于对比学习的视觉-语言模型在医学图像报告对齐任务中展现出强大的零样本诊断能力,但3D医学影像训练批次构成对表征学习的影响尚未明确。本文复现了Merlin模型,该模型使用对称InfoNCE损失对齐3D腹部CT与放射科报告,在30种病灶上的零样本宏F1达到74.45%(原论文为73.00%)。研究考察两个变量:首先在全数据集上通过分段级均衡采样控制批次中正常与异常比例为25:75、50:50、75:25,所有平衡配置均低于未平衡基线2.4至2.8个百分点,其中75:25表现最佳(72.02%);其次在4,362例子集上进行数据缩放实验,分别使用20%、40%和100%数据训练,性能从65.26%升至71.88%,且各病灶对数据敏感度差异显著。在同一子集上强制50:50平衡采样进一步导致性能下降至68.01%,表明无论数据规模或平衡粒度,显式类平衡均会损害表现。结果表明,小批量下随机采样的固有多样性,结合Merlin在解剖子区间的交替批处理机制,比人工设定类别比例提供更有效的正则化。

原文摘要 · Abstract (English)

Vision-language models trained with contrastive learning on paired medical images and reports show strong zero-shot diagnostic capabilities, yet the effect of training batch composition on learned representations remains unexplored for 3D medical imaging. We reproduce Merlin, a dual-encoder model that aligns 3D abdominal CT volumes with radiology reports using symmetric InfoNCE loss, achieving a zero-shot macro F1 of 74.45% across 30 findings (original: 73.00%). We then investigate two axes of variation. First, we control the normal-to-abnormal ratio within training batches at 25:75, 50:50, and 75:25 using section-level balanced sampling on the full dataset. All three configurations underperform the unbalanced baseline by 2.4 to 2.8 points, with 75:25 achieving the best result (72.02%) among balanced variants. Second, we conduct data scaling ablations on a 4,362-study subset, training with 20%, 40%, and 100% of the data. Performance scales sub-linearly from 65.26% to 71.88%, with individual findings varying dramatically in data sensitivity. Enforcing 50:50 balanced sampling on the same subset further degrades performance to 68.01%, confirming that explicit class balancing hurts regardless of dataset or balancing granularity. Our results indicate that the stochastic diversity of random sampling, combined with Merlin's alternating batching over anatomical subsections, provides more effective regularization than engineered class ratios at the small batch sizes required by 3D medical volumes.

医学影像零样本学习视觉语言模型数据采样

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。