构建首个全身PET/CT多模态图文数据集，支持功能与结构联合推理。
MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT
- 融合代谢、解剖与语义标注，构建490例全身3D影像数据集
- 包含5万+器官分割掩码与10万+跨模态问答对，支持精准诊断推理
- 适合医学AI研究者、放射科医生及多模态模型开发者使用
全身PET/CT的代谢功能与解剖结构联合解读对临床诊断至关重要。尽管3D医学视觉语言模型取得进展，现有工作仍局限于局部CT成像，缺乏对全身PET/CT的系统分析。本文提出MetaStructAtlas，一个大规模地面实况全身PET/CT多模态数据集，整合了多模态影像与解剖、代谢、语义标注。该数据集包含490对配准的3D PET与CT图像，50,470个器官级分割掩码及对应的地面实况放射科报告。为支持交互式推理，我们进一步构建了MetaStructVQA基准，包含100,565个跨模态问答对，明确关联诊断问题与多模态视觉证据，涵盖解剖、形态与代谢特征。最后，我们在MetaStructVQA上评估了先进3D医学视觉语言模型，为核医学中的多模态表征学习与全身整合推理奠定基础。
原文摘要 · Abstract (English)
The joint interpretation of metabolic function and anatomical structure is essential for clinical diagnosis in whole-body PET/CT. Although recent advances in 3D medical vision-language models have demonstrated remarkable progress, current efforts are limited to regional CT imaging, leaving a critical void in comprehensive whole-body PET/CT analysis. In this work, we introduce MetaStructAtlas, a large-scale dataset for grounded whole-body PET/CT interpretation that synthesizes multimodal imaging with integrated anatomical, metabolic, and semantic annotations. MetaStructAtlas provides 490 co-registered 3D PET and CT volumes with 50,470 organ-level segmentation masks and grounded radiology reports. To facilitate interactive reasoning, we further developed MetaStructVQA, a standardized 3D grounded visual question-answering benchmark containing 100,565 QA pairs. This framework explicitly links diagnostic queries to visual evidence across modalities, encompassing anatomical, morphological, and metabolic characteristics. Finally, we evaluate state-of-the-art 3D medical VLMs on MetaStructVQA, establishing a robust foundation for multimodal representation learning and integrated whole-body reasoning in nuclear medicine.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。