XMedFusion通过多智能体协作提升医学影像报告生成的准确性与可解释性。
XMedFusion: A Knowledge-Guided Multimodal Perception and Reasoning Framework for Autonomous Medical Systems

- 分模块设计视觉感知、知识图谱构建与检索引导报告生成
- 在胸部X光数据集上多项指标显著优于基线模型
- 适合追求高可靠性与透明度的自主医疗系统研发者
自主医疗与机器人系统日益依赖智能感知与推理能力来解读视觉数据并支持临床决策。放射科报告生成是此类自动化诊断流程的关键环节,但现有端到端多模态模型常因视觉定位薄弱导致解释不可靠,遗漏细微临床发现。本文提出XMedFusion,一种模块化AI框架,作为自主医疗系统的智能感知与推理模块。该框架将视觉信息分解为协同的功能组件,模拟专家分析过程:包括提取图像证据的视觉感知代理、构建临床相关发现的知识图谱代理,以及确保报告结构一致的检索引导起草过程。一个合成代理通过推理驱动验证,迭代整合视觉与结构化证据,生成可靠且可解释的诊断输出。在公开胸部X光数据集上的实验表明,相比基线视觉语言模型,其在BLEU-1(0.0493→0.3359)、ROUGE-L(0.0863→0.2440)、METEOR(0.0829→0.1708)等指标上均有显著提升,并在语义评估指标上表现优异,如一致性(2.38→7.80)和准确率(2.34→6.93)。结果表明,结构化多智能体感知与推理能有效提升智能医学影像系统的鲁棒性、透明度与自动化水平,适用于自主医疗与机器人诊断工作流集成。
原文摘要 · Abstract (English)
Autonomous medical and robotic systems increasingly rely on intelligent perception and reasoning capabilities to interpret visual data and support clinical decision making. Radiology report generation represents a critical component of such automated diagnostic workflows, yet existing end-to-end multimodal models often suffer from weak visual grounding, resulting in unreliable interpretations and omission of subtle clinical findings. This paper presents XMedFusion, a modular AI framework designed as an intelligent perception and reasoning module for autonomous medical systems. The proposed framework decomposes visual information into coordinated functional components that emulate expert-driven analysis, including a visual perception agent that extracts image-grounded evidence, a knowledge graph construction agent that structures clinically relevant findings, and a retrieval-guided drafting process that ensures a consistent reporting structure. A synthesis agent iteratively integrates visual and structured evidence through reasoning-driven verification to produce reliable and interpretable diagnostic outputs. Experimental evaluation on a public chest radiograph dataset demonstrates significant improvements over baseline vision-language models, achieving gains from 0.0493 to 0.3359 in BLEU-1, 0.0863 to 0.2440 in ROUGE-L, and 0.0829 to 0.1708 in METEOR, along with substantial improvements in semantic evaluation metrics such as Consistency (2.38 to 7.80) and Accuracy (2.34 to 6.93). The results highlight the effectiveness of structured multi-agent perception and reasoning for enhancing robustness, transparency, and automation in intelligent medical imaging systems, enabling integration into autonomous healthcare and robotic diagnostic workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。