arXiv:2606.08420cs.CV2026-06

让AI理解胸部X光片的解剖结构,提升定位精度。

CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs

论文配图:CheXanatomy: Anatomy-Aware Vision-Language Modeling for Chest Radiographs
图 1 · 摘自论文原文
  • 用自回归预测生成解剖分割图,直接注入解剖知识
  • 在真实X光片上表现接近专用模型,且更抗分布偏移
  • 小样本下适应新任务更快,适合医疗精准诊断场景

视觉语言模型(VLM)虽具备强大图像理解能力,但主要优化全局对齐,缺乏细粒度解剖结构建模,难以胜任分割等空间精确任务。本文提出CheXanatomy框架,通过自回归标记空间监督,将显式解剖知识融入预训练VLM。模型不依赖特定任务解码头,而是通过下一标记预测生成解剖分割掩码。为实现可扩展标注,利用CT体积合成逼真胸片,并将CT分割标签前向投影至2D获得一致掩码。在合成与真实胸片数据上评估,相比U-Net基线,自回归解剖监督在分布内表现相当,在域偏移至真实CXRs时展现更强几何鲁棒性。此外,经解剖预训练的模型在有限监督下适应新定位任务时样本效率更高。更大模型和更高输入分辨率提升性能,而视觉编码器微调影响有限。结果表明,将解剖结构嵌入生成目标可促进空间感知表示,支持解剖感知的医学视觉语言建模。

原文摘要 · Abstract (English)

Vision-language models (VLMs) pretrained on large-scale image-text pairs demonstrate strong image-level understanding, but are primarily optimized for global alignment and do not explicitly encode fine-grained anatomical structure, limiting their suitability for spatially precise tasks such as segmentation. We introduce CheXanatomy, a framework that integrates explicit anatomical knowledge into a pretrained VLM through autoregressive token-space supervision. Instead of adding task-specific decoder heads, the model is trained to generate anatomical segmentation masks via next-token prediction. To enable scalable supervision, we synthesize realistic chest radiographs from CT volumes and forward-project CT segmentation labels to obtain anatomically consistent 2D masks. We evaluate the approach on synthetic and real chest radiographs against a U-Net baseline, including ablations on model scale, input resolution, and vision encoder fine-tuning. Autoregressive anatomical supervision achieves performance comparable to specialized convolutional models in-distribution and demonstrates improved geometric robustness under domain shift to real CXR data. In addition, anatomy-pretrained models exhibit improved sample efficiency when adapting to novel localization tasks under limited supervision. Larger models and higher input image resolution improve performance, while vision encoder fine-tuning has limited effect. These results show that embedding anatomical structure directly into the generative objective promotes spatially grounded representations and supports anatomy-aware medical vision-language modeling.

医学影像解剖结构视觉语言模型分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。