医学影像模型只记住了器官的常见位置，不会在患者体内比较结构关系。
Do Medical Vision Models Reason About Anatomy? Probing the Spatial Inductive Biases of Learned Visual Representations
- 设计八项探针测试器官位置、关系和空间推理能力
- 跨域零样本迁移时准确率降至随机水平，说明依赖记忆而非计算
- 用全局池化头会高估模型推理能力，真实能力极低
解读CT扫描需要比较双侧结构、判断器官间距并识别其正常位置。当前医学视觉编码器主要通过诊断准确率或集成多模态系统评估，故障难以归因，因此其表征是否支持此类空间推理仍不明确。我们构建了SPAR-Bench，包含八项针对腹部多器官CT的探针，分离坐标定位、关系推理与空间查询任务，并应用于五种架构配置及三类医学基础模型（冻结与微调）。要求在切片内进行比较的探针始终处于随机水平，无论预训练规模、微调或架构均无法缩小差距。在本域中看似解决的探针，在零样本迁移下准确率跌至随机水平，表明其准确性源于对标准解剖结构的记忆，而非图像上的计算。使用全局池化头而非完整标记序列读取冻结特征，使关系恢复准确率从0.7%提升至67.8%，说明池化探针严重低估了表征潜力。能被编码器正确回答的问题，四个开源多模态大模型（MLLMs）却只能随机作答。结果表明，这些编码器仅携带器官通常位置的地图，缺乏在特定患者体内比较结构的推理机制。代码与数据将公开于https://spar-bench.github.io。
原文摘要 · Abstract (English)
Interpreting a CT scan means comparing structures on either side, judging how far apart organs sit, and knowing where each one belongs. Medical vision encoders are evaluated on diagnostic accuracy, or through assembled multimodal systems where a failure is hard to attribute, so it remains unclear whether their representations support any of this. We construct SPAR-Bench, eight probes over multi-organ abdominal CT that separate coordinate localization, relational reasoning, and spatial queries, and apply them to five architectural configurations and three medical foundation models, frozen and finetuned. Probes that ask for a comparison within the slice stay at chance, and neither pretraining scale, finetuning, nor architecture closes the gap. Probes that appear solved in domain fall to chance under zero-shot transfer, indicating that their accuracy reflects recall of canonical anatomy rather than computation over the image. Reading the same frozen features with a pooled head rather than the full set of tokens moves relational recovery from 0.7% to 67.8%, so pooled probing understates what a representation holds. Questions the encoders answer well are answered at chance by four open-weight MLLMs. Our results suggest these encoders carry a map of where organs usually lie, and little of the machinery for comparing structures within a particular patient. Code and data will be available at https://spar-bench.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。