多模态MRI数据融合生成脊柱影像报告,提升诊断效率与可解释性。
A multi-agent system for spine MRI report generation from multi-sequence imaging

- 构建多智能体系统,通过双编码器融合T1/T2序列信息并生成患者级嵌入
- 在32,047例患者数据上训练,实现跨设备、跨队列的强泛化能力
- 支持病灶定位与图文检索,适合临床辅助诊断与医学AI研究
脊柱病变是全球疼痛和残疾的主要原因。脊柱MRI在临床评估中至关重要,但其解读复杂且耗时,需整合多序列和解剖区域的信息。尽管自动化MRI分析取得进展,如何有效融合多序列数据同时保留序列特异性诊断信息仍是挑战。本文提出SpineAgent,一个基于32,047名患者、453,683个MRI序列(共13,441,191张切片)训练的多序列基础模型的多智能体框架。首先分别在T1和T2序列上预训练两个DINOv3编码器;随后引入持续学习策略,利用T1/T2编码器合成其他序列的嵌入,生成跨序列的患者级表示。基于此嵌入,SpineAgent达到当前最优性能,并在跨制造商和跨队列评估中表现出强泛化能力。除分类外,还能定位病灶相关切片并分割病灶区域。支持多模态图像-报告检索,为可扩展、可解释的报告生成奠定基础。我们将验证后的功能集成到37个专用智能体中,最终将输出作为结构化标记输入至端到端训练的医学报告智能体。通过自动指标及五位放射科专家评估,SpineAgent在脊柱MRI报告生成任务中表现领先。
原文摘要 · Abstract (English)
Spinal pathology is a leading cause of pain and disability worldwide. Spine MRI is central to clinical evaluation, yet its interpretation remains complex and time-consuming, requiring integration of information across multiple imaging sequences and anatomical regions. Despite recent advances in automated MRI analysis, effectively combining multi-sequence data while preserving sequence-specific diagnostic information remains an open challenge. Here we present SpineAgent, a multi-agent framework for spine MRI report generation built upon a multi-sequence foundation model trained on routine clinical data from 32,047 patients and 453,683 MRI series, comprising a total of 13,441,191 MRI slices. To accommodate diverse modalities of sequences, we first pre-train two DINOv3-based encoders separately on T1- and T2-weighted sequences. We then introduce a continual training strategy that learns a synthesizer to embed images of other sequences using the T1 and T2 encoders, producing patient-level embedding that integrates various signals across MRI sequences. Using these embeddings, SpineAgent achieves state-of-the-art performance, and demonstrates strong generalizability under cross-manufacturer and cross-cohort evaluation. Beyond classification, SpineAgent enables pathology localization by identifying findings-relevant slices and segmenting pathological regions. It also supports multimodal image-report retrieval, providing a solid foundation for scalable and explainable MRI report generation. We further integrate these validated capabilities of SpineAgent into 37 specialized agents. Finally, we incorporate their outputs as structured tokens within a Medical Report Agent trained end-to-end for report generation. Through both automated metrics and expert evaluation by five radiologists, SpineAgent achieves leading performance in spine MRI report generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。